-- Logs begin at Wed 2022-05-25 13:03:01 UTC, end at Wed 2022-05-25 13:56:11 UTC. -- May 25 13:36:28.466490 dv-cinder-803823-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 13:36:28.474151 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 13:36:28 2022] *** May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: nodename: dv-cinder-803823-8-iscsi May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: machine: x86_64 May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: clock source: unix May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: pcre jit disabled May 25 13:36:28.476161 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: detected number of CPU cores: 4 May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: current working directory: / May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: detected binary path: /usr/bin/uwsgi-core May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: your processes number limit is 47680 May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: your memory page size is 4096 bytes May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: detected max file descriptor number: 2048 May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: lock engine: pthread robust mutexes May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: thunder lock: enabled May 25 13:36:28.476758 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 13:36:28.481547 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 13:36:28.522903 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: Python main interpreter initialized at 0x559b5afe29c0 May 25 13:36:28.522903 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: python threads support enabled May 25 13:36:28.522903 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: your server socket listen backlog is limited to 100 connections May 25 13:36:28.522903 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: your mercy for graceful operations on workers is 90 seconds May 25 13:36:28.523282 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: mapped 403077 bytes (393 KB) for 2 cores May 25 13:36:28.523358 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: *** Operational MODE: preforking *** May 25 13:36:28.523641 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: *** uWSGI is running in multiple interpreter mode *** May 25 13:36:28.523641 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: spawned uWSGI master process (pid: 91815) May 25 13:36:28.523858 dv-cinder-803823-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 13:36:28.524730 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: spawned uWSGI worker 1 (pid: 91825, cores: 1) May 25 13:36:28.524841 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: spawned uWSGI worker 2 (pid: 91826, cores: 1) May 25 13:36:28.524907 dv-cinder-803823-8-iscsi devstack@keystone.service[91815]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 13:36:34.219169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:36:34.219169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: from cryptography.utils import int_from_bytes May 25 13:36:34.219973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:36:34.219973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: from cryptography.utils import int_from_bytes May 25 13:36:34.259890 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:36:34.259890 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: from cryptography.utils import int_from_bytes May 25 13:36:34.260515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 13:36:34.260515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: from cryptography.utils import int_from_bytes May 25 13:36:34.274303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 13:36:34.274303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 13:36:34.274303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] command line args: [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 13:36:34.274303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 13:36:34.274303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ================================================================================ {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 13:36:34.274732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] admin_token = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.274732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] config_dir = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.274918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.275054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] config_source = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.275232 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] debug = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.275427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.275794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.276339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.276339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.276521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.276685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.276864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_config_append = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.277027 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.277187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_dir = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.277363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_file = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.277699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.278487 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.278693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.278912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.279090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.279258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.279504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.279751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.279984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.280218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.281971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.21/identity {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.282149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] publish_errors = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.282385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.282615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.282790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.283034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.283266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.283498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] standard_threads = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] use_journal = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] use_json = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] use_stderr = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] use_syslog = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.285745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.285745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.285745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.286065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.286323 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.286563 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.287016 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.287318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.287577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.287823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.288080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.288318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.288583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.288823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.289072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.289321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.289559 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.289811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.290063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.290308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.290543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.290796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.291055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.291309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.291558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.291841 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.292079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.292314 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.292549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.292820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.293129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.293468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.293703 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.293952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.294201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.294437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.294684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.294916 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.295149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.295383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.295612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.295839 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.296069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.296298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.296540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.296805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.297035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.297268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.297497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.297730 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.297961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.298206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.298436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.298671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.298904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.299136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.299368 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.299601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.299829 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.300075 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.300306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.300540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.300794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.301051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.301284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.301514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.301749 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.301985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.302256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.302492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.302982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.303216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.303450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.303685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.303931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.304166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.304399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.304666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.304902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.305138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.305370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.305601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.305834 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.306093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.306332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.306570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.306804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.307043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.307275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.307507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.307739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.307966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.308193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.308421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.308701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.309119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.309556 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.309991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.310434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.310847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.311175 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.311445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.311691 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.311927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.external = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.312165 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.312288 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 13:36:34.312406 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 13:36:34.312525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.312666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] command line args: [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 13:36:34.312864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.312990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 13:36:34.313112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.password = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.313228 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ================================================================================ {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 13:36:34.313341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] auth.token = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.313460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] admin_token = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.313586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.313701 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] config_dir = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.313814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.313926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.314039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.314156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] config_source = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.314289 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.314405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] debug = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.314519 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.314632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.314785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.314901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.314901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.315111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.315111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.315321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.315445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.315559 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.315559 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.315768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.315881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_config_append = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.315996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.316108 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.316205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.316307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_dir = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.316307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_file = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.316484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.316648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.316764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.316862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.316968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.317062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.317162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.317258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.317367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.317484 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.317597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.317699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.317811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.317953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.318081 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.318196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.318310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.318431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.318552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.318674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.318789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.318892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.319003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.319122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.319232 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.319353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.319469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.319571 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.319682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.319801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.319918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.319918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.320161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.21/identity {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.320287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.320402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] publish_errors = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.320516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.320646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.320772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.320879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.320879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.321090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.321090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.321301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.321301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.321525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.321638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] standard_threads = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.321751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.321862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.321975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.322095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.322217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.322331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.322444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.322558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] use_journal = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.322679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.322794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] use_json = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.322794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] use_stderr = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.323015 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] use_syslog = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.323247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 13:36:34.323483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323716 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.323943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.324063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.324178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.324825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.324951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325380 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325680 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.325926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.326887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327116 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327816 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.327933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328047 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328869 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.328982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.329947 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330761 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330761 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.330974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331839 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.331953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.332926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333167 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.333977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334555 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.334897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.335856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.336973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337455 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.337930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338047 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338483 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.338952 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.339847 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.340026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.340735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.340869 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.340985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.341953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.342895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343364 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.343956 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.344933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345406 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345636 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.345996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346454 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] role.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346695 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] role.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.346932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347857 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.347980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348555 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.348935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.349977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.external = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350534 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.350916 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.password = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351245 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] auth.token = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351841 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.351956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352070 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352193 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.352948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353181 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353650 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.353994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354232 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354815 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.354938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355514 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.355894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356126 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356241 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.356958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.357894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358015 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358600 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.connection = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.358961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.359891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360806 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.360923 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361514 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361631 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.361959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.362895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91826) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 13:36:34.363238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.363770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.364086 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.364208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.364419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.364691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.364937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.365180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.365431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.365665 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.365915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.366162 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.366401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.366649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.366898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.367130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.367368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.367606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.367845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.368082 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.368326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.368578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.368825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.369075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.369381 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.369673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.369953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.370206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.370456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.370694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.370933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.371199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.371456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.371701 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.371961 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.372195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.372444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.372725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.372976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.373214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.373446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.373683 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.373920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.374158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.374504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.374764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.375021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.375275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.375528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.375780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.376018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.376269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.376516 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.376788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.377031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.377270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.377524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.377776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.378017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.378276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.378542 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.378758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.378999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.379238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.379477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.379727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.379973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.380211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.380452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.380718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.380973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.381225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.381466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.381721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.381975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.382212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.382461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.382698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.382940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.383177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] role.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.383426 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] role.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.383660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.383911 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.384162 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.384412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.384689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.384941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.385190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.385440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.385691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.385926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.386175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.386425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.386674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.388449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.388984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.389235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.389497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.389758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.390014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.390268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.390509 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.390769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.391025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.391268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.391505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.391744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.391980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.392234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.392494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.392783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.393077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.393269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.393445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.393623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.393808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.394012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.394012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.394012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.394532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.394532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.394903 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.395095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.395280 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.395468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.395654 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.395859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.396101 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.396342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.396735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.396937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.397127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.397311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.397514 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.397829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.398090 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.connection = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.398341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.398595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.398840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.399084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.399351 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.399629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.399972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.400357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.400811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.401290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.401709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.402136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.402231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.403230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.403539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.403849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.404270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.404725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.405172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 13:36:34.405486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91825) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 13:36:34.431099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.431333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.431442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.431563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.431647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.442307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.461578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.461810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.485236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.500269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.536374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.536624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.538030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.538270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.603417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.603649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.603791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.603952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604495 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.604950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.628995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629221 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629846 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.629926 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.630001 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.630074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.630148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.630230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.630306 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.641821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.642076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.654917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.655964 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91826) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.656850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91825) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 13:36:34.674460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.674583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.674662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:36:34.674754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:36:34.674997 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:36:34.675286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 13:36:34.675466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.675979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.676062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.676232 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.676315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.687351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.687623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.687886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.688162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.689107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.689386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.689653 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.689932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.696228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.698096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.699701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.699988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.700263 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.700535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.700840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.701098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.701362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.701519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.701735 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.701817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.702963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.703237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.703511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.703780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.704053 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.704324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.729704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.732194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.732472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.732556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.734339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.734626 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.738007 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.738238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.738517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.740054 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.740348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.740661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.745872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.746198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.748141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.748426 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.750675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.751079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.751364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.751642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.751917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.752191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.752459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.752750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.752985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.753398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.753686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.753971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.754250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.754528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.754801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.755070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.771504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.771774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.773817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.774096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.776438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.779314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.780746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.781040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.783024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.783320 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.786098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.788376 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.789697 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.792073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.794123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.794599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.794885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.795168 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.795459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.795741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.796592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.796887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.797181 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.797474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.797757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.798048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.814878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.815152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.815413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.815671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.817040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.817328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.817592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.817855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.823646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.825925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.826193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.826807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.828517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.828830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.829264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.829540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.829816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.830091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.830370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.831914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.832197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.832481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.832788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.833063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.844552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.844846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.845127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.845403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.845675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.845942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.846217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.847414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.847694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.847973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.848260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.848538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.848847 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.849130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.864194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.867148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.867465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.870459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.870792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.872532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.873836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.874186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.874464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.874738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.875458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.877106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.877394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.877686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.882114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.883727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.885185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.887319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.887587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.887883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.888166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.888425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.890730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.891019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.891307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.891591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.897529 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.897817 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.898212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.900755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.901048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.901441 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.906715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91826) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.906992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.907267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.907555 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.907824 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.908095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.908373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.908673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.908949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.909212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.909473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.909736 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.910002 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.910193 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91825) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 13:36:34.910278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91826) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.910473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.910751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.911024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.911295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.911573 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.911855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.912149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.912419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.912723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.912994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.913253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.913524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.913792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91825) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 13:36:34.948992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559b5afe29c0 pid: 91826 (default app) May 25 13:36:34.952951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fdab3-d5bc-4774-8887-a7b2985879ce None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:34.953244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fdab3-d5bc-4774-8887-a7b2985879ce None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:34.953382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559b5afe29c0 pid: 91825 (default app) May 25 13:36:34.953523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fdab3-d5bc-4774-8887-a7b2985879ce None None] PATH_INFO: `/v3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:34.957789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 1/1] 10.222.0.21 () {50 vars in 791 bytes} [Wed May 25 13:36:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:43.033945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:43.034526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:43.034526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:43.038960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 1/2] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:36:43.045012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:43.045464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:43.045557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:43.114511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: INFO dbcounter [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] Registered counter for database keystone May 25 13:36:43.136137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG oslo_db.sqlalchemy.engines [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91826) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 13:36:43.142468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writer thread running {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 13:36:43.327174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG passlib.handlers.bcrypt [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91826) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 13:36:43.327527 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG passlib.handlers.bcrypt [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91826) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 13:36:43.383989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:43.386140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:43.512651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 2/3] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 13:36:43.669336 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:43.738081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: INFO dbcounter [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] Registered counter for database keystone May 25 13:36:43.761796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG oslo_db.sqlalchemy.engines [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91825) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 13:36:43.768313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writer thread running {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 13:36:43.888172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d7a93-be53-414f-9ee2-5c437a53735a None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:43.892728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:43.893403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:43.893692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:43.893973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:43.902896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:43.954817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING keystone.common.rbac_enforcer.enforcer [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 13:36:43.955159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c725a-9032-4d63-a209-0ed688833289 admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:43.965870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 2/4] 10.222.0.21 () {58 vars in 1324 bytes} [Wed May 25 13:36:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 13:36:43.971899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:43.986297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608c703-6432-4a3a-a295-69b74162e464 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:43.990748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:43.991863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:43.992154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:43.992442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] PATH_INFO: `/v3/projects/d26306eca6ff451eb60119f9e4cf5383` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:43.994004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] RBAC: Authorizing `identity:get_project(project_id=d26306eca6ff451eb60119f9e4cf5383)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:44.153268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING keystone.common.rbac_enforcer.enforcer [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 13:36:44.153890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35fa5-ac05-4990-bffc-cd11375997d2 admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:44.158274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 3/5] 10.222.0.21 () {56 vars in 1207 bytes} [Wed May 25 13:36:43 2022] GET /identity/v3/projects/d26306eca6ff451eb60119f9e4cf5383 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:46.380026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f68d0c-84a5-4b52-ad1a-e594fdce5b5f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:46.380632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f68d0c-84a5-4b52-ad1a-e594fdce5b5f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:46.380632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f68d0c-84a5-4b52-ad1a-e594fdce5b5f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:46.384810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 3/6] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:46.390329 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:46.390770 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:46.390897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:46.410239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:46.411764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:46.420388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 4/7] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 13:36:46.442593 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f68d0c-84a5-4b52-ad1a-e594fdce5b5f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:46.455196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f68d0c-84a5-4b52-ad1a-e594fdce5b5f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:46.459521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:46.460184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:46.460474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:46.460782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:46.470365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:46.472185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea2258-044c-4d08-b387-b4b426461d75 admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:46.481660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 4/8] 10.222.0.21 () {58 vars in 1324 bytes} [Wed May 25 13:36:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 13:36:46.487551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:46.497496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5820-1bd3-4134-9856-a5b34d82feb5 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:46.501910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:46.502779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:46.503066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:46.503344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:46.504776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:46.506623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e081e9b-356b-47bf-b847-b21f782a74e6 admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:46.508209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 5/9] 10.222.0.21 () {56 vars in 1198 bytes} [Wed May 25 13:36:46 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:48.793384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4fc0-f0c1-40b4-8d29-527011138734 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:48.793920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4fc0-f0c1-40b4-8d29-527011138734 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:48.793920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4fc0-f0c1-40b4-8d29-527011138734 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:48.795358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 5/10] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:48.800717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:48.801170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:48.801409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:48.818410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:48.819579 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:48.847685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1f149-1294-4a18-8c12-565bc042c271 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:48.848196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1f149-1294-4a18-8c12-565bc042c271 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:48.848479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1f149-1294-4a18-8c12-565bc042c271 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:48.849872 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 6/11] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:36:48.855215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:48.855527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:48.855803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:48.896466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG passlib.handlers.bcrypt [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91825) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 13:36:48.896912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG passlib.handlers.bcrypt [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91825) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 13:36:48.900245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 6/12] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:48.923114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:48.943711 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:48.945365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:48.946892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.031369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c5a3f-9718-4506-a767-0f4b3e98fbad None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.056583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 7/13] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:36:49.073484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.074287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.074580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.074865 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.086161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.086414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:36:49.098320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:36:49.099376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:36:49.100392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 7/14] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:36:49.106092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.187582 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.194924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027a9e3-d8ef-4664-88bc-8f451466e8f9 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.197289 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.207387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c2201-bb81-42f2-9253-fe0503e049f5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.234357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.235263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.235552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.235843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.242388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.243058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.243342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.243615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.244400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:49.244891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.246517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.258216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 8/15] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:49.264615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.304013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.305556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.367017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea7bba-031f-4b71-9d19-64130f8cfa4b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.420993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 8/16] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:36:49.426564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.429314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.437608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.438505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.438785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.439065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.440325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.441845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.443473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 9/17] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:49.449626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.464303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.531202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002b6e-61aa-4dc1-bdb0-1330774f0622 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.550574 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c838f71-c940-4940-bf90-c83d200fba0c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.565692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.566491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.566775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.567059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.585115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.585929 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.586220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.586503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.586778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.587719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.588398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.589585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.591640 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 10/18] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:49.597381 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:36:49.609617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:36:49.610522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:36:49.611312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:36:49.611312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:36:49.611312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:36:49.611312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:36:49.611312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 9/19] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:36:49.616596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:49.677643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.699643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822de93-2dbe-4e18-b126-3a52c134b4fb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.699913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:49.735949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.736658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.736948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.737229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.738098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.domain.id=default, user.id=4118f6e846e744a8a962f12a116b5928) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:49.738419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.739935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdca0a2-841c-4026-80ba-1c9d30ae064e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.749300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 11/20] 10.222.0.21 () {56 vars in 1322 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/role_assignments?user.id=4118f6e846e744a8a962f12a116b5928&scope.domain.id=default&role.id=698416d22e5b4c278ff780c5eef36ede => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:49.876277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515225-bb7d-40e9-999c-f355e38b761d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:49.912614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:49.913295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:49.913583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:49.913866 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:49.914607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:49.914921 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:49.920340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6a4b2-7a67-44a4-b81e-cf67d4869479 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:49.956673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 10/21] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:52.206098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45624-0e24-4c67-8e55-43bf94d21eb9 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.206646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45624-0e24-4c67-8e55-43bf94d21eb9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.206646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45624-0e24-4c67-8e55-43bf94d21eb9 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.208172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 12/22] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:52.226462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.227415 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.227919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.252110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:52.253678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:52.338446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 11/23] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:52.392971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45624-0e24-4c67-8e55-43bf94d21eb9 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:52.538453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45624-0e24-4c67-8e55-43bf94d21eb9 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:52.592608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:52.595840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.595840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.595840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.617922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-00fb3fd0-1c69-4b09-bbeb-18218eb0065e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:36:52.626229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:36:52.627071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:36:52.627816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 13/24] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:36:52.633464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:52.706332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:52.785538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7049b27-d014-4241-b417-389b8f304ecf None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:52.828856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544937-9146-4f89-8822-b49491e07272 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.829131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544937-9146-4f89-8822-b49491e07272 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.829354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a544937-9146-4f89-8822-b49491e07272 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.830751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 14/25] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:52.841838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:52.842532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.842816 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.843090 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.843855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:52.844159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:52.845592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048c01-f0af-4b16-934f-e905586cb609 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:52.863775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 12/26] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:52.868447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a544937-9146-4f89-8822-b49491e07272 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:52.869618 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e6a-5d9d-4323-b87f-b8a69d91b52b None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:52.870093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e6a-5d9d-4323-b87f-b8a69d91b52b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:52.870590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e87e6a-5d9d-4323-b87f-b8a69d91b52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:52.876818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:52.901401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-43e87e6a-5d9d-4323-b87f-b8a69d91b52b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:52.902638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-43e87e6a-5d9d-4323-b87f-b8a69d91b52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:53.025140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a544937-9146-4f89-8822-b49491e07272 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.070822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 13/27] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:36:53.074086 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1682b-658c-4161-851e-3d9fdbfda5d4 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.075493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1682b-658c-4161-851e-3d9fdbfda5d4 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.076290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1682b-658c-4161-851e-3d9fdbfda5d4 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.078753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.079862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.080533 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 14/28] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:53.081809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3558f-3bb2-4edb-964b-8a00e832af79 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.082285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.082717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.083677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3558f-3bb2-4edb-964b-8a00e832af79 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.084107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3558f-3bb2-4edb-964b-8a00e832af79 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.085029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.086531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 15/29] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:53.088348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:53.091593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 15/30] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:36:52 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:53.091827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bb2a3-623d-423f-9246-1d7eff428272 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.092273 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bb2a3-623d-423f-9246-1d7eff428272 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.092759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565bb2a3-623d-423f-9246-1d7eff428272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.101828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:53.125432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-565bb2a3-623d-423f-9246-1d7eff428272 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:53.128607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-565bb2a3-623d-423f-9246-1d7eff428272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:53.175437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:53.225063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:53.264097 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 16/31] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:53.265134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07621450-a03f-4b86-a860-3b16291e6e88 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.269071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.270420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.271312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.300994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:53.302655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:53.343828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.344580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.344872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.345336 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.347523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.349543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:53.351853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 16/32] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:53.358979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:53.411337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 17/33] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:53.414629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:53.524170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17a25e-d157-46f6-8062-ff025dddc81a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.582856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.585500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.587414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.587945 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.603935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:53.608039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.609737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:53.612149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:53.617537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5c4b2-d63f-4a2b-885f-fea8da25f1d6 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:36:53.630284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:36:53.631127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:36:53.631970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:36:53.631970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:36:53.631970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:36:53.631970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:36:53.634045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 17/34] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:36:53.635131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:53.659396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.660764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.661233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.661656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.662802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.664890 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:53.753601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d1030-ddc0-41cd-b885-fee4dbcdbaff None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.788848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.790017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.790459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.791498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] PATH_INFO: `/v3/domains/default/users/4118f6e846e744a8a962f12a116b5928/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.793975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4118f6e846e744a8a962f12a116b5928, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.795781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:53.818547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 18/35] 10.222.0.21 () {60 vars in 1131 bytes} [Wed May 25 13:36:53 2022] POST /identity/v3/roles => generated 225 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:36:53.819969 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:53.948955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d28ea-7b5e-44c7-ae55-d5db399f4046 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:53.990965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:53.992026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:53.992511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:53.993055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:53.995346 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:53.997748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e02469-f609-44f6-bddb-66f89addb117 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:54.005768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 18/36] 10.222.0.21 () {58 vars in 1361 bytes} [Wed May 25 13:36:53 2022] PUT /identity/v3/domains/default/users/4118f6e846e744a8a962f12a116b5928/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:36:54.007002 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:54.016977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 19/37] 10.222.0.21 () {60 vars in 1131 bytes} [Wed May 25 13:36:53 2022] POST /identity/v3/roles => generated 219 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:36:54.018960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b45b3-89fb-4cec-8c86-0beac00d376f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.019448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b45b3-89fb-4cec-8c86-0beac00d376f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.019977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b45b3-89fb-4cec-8c86-0beac00d376f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.021803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 20/38] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:54.031377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.032087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.032656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.073884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:54.075537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:54.108989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:54.114139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:54.195322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 21/39] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:54.205506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cef439-4c07-44a4-b46e-044c7cc42cc9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:54.223408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:54.253113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:54.253944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.254241 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.254690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.255463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:54.256157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:54.258346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:54.285691 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 19/40] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:36:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:54.293658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:54.365655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b53335-1acf-4f7a-9aca-d22e5b058c59 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:54.416952 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:54.418048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.418513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.418993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.419908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:54.421490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc8fee-6252-478e-a154-edacaccde38a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:54.444902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 22/41] 10.222.0.21 () {60 vars in 1131 bytes} [Wed May 25 13:36:54 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:36:54.474546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbbe45-451b-452d-a0c0-343aab2044c5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:54.496158 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:54.522597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:54.523292 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.523576 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.523910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.524777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:54.526432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced97e0-a2c9-4479-8b36-3b6a01118cbc None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:54.563666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 20/42] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:36:54 2022] POST /identity/v3/projects => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:36:54.608042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411db44-ac24-4007-97d0-f9225e374e25 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.608341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411db44-ac24-4007-97d0-f9225e374e25 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.608818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411db44-ac24-4007-97d0-f9225e374e25 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.610642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 23/43] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:54.616765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.617138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.617433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.637787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:54.639462 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:54.722510 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 21/44] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:54.746562 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411db44-ac24-4007-97d0-f9225e374e25 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:54.758629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:54.859098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411db44-ac24-4007-97d0-f9225e374e25 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:54.904068 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:54.904950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:54.905248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:54.905537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:54.907186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:54.908793 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7be818-87ac-4609-b175-8461ae9de013 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:54.910576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 24/45] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:54.916748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:55.012506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:55.019399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8583717-0681-4580-8e32-b18905a3ab61 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:55.055012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:55.055701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:55.055993 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:55.056867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:55.057709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:55.059239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ea271-6f87-4058-8423-28ad1cef6ce3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:55.086444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 22/46] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:36:54 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:36:56.688422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8c4de-844d-463d-9884-0da990990fba None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:56.688970 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8c4de-844d-463d-9884-0da990990fba None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:56.689083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8c4de-844d-463d-9884-0da990990fba None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:56.690394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 25/47] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:36:56.696700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:56.697360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:56.697511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:56.714986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:56.716203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:56.789552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 23/48] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:36:56.814283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8c4de-844d-463d-9884-0da990990fba None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:56.896276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:56.926223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8c4de-844d-463d-9884-0da990990fba None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:56.961955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:56.963285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:56.963571 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:56.963855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:56.976079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:36:56.985012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:36:56.985881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:36:56.986643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 26/49] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:36:56.993847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.071584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:57.101036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566b8ec-45a2-4f8f-8a76-f1d4edb9eb73 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.141388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.142107 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.142399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.142680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.143474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:57.143792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.145306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.156951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 24/50] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:57.164084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.273182 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37bcc6-5368-4d75-aec3-28470f5e9164 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.311556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.312447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.312783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.313075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.314566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.316072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f871-167a-4aaa-b85d-36d89ab86b16 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.317773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 27/51] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:36:57 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:57.323722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.324210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79194-7079-4c28-ad8f-9299ad023764 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.324515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79194-7079-4c28-ad8f-9299ad023764 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.324833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be79194-7079-4c28-ad8f-9299ad023764 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.326190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 28/52] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:36:57.333507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.333806 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.334096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.341083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:57.359427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:36:57.360672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:36:57.430748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c6906-6e8f-453b-8457-0a93ac8939e0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.431647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 29/53] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:36:57.459799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.471808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.472653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.472951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.473233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.474434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.475933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.478094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 25/54] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:36:57.484179 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.494593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:57.565112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900b2e7-f511-47ef-afdf-fa238809b19f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.599022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa0b8-f8f1-40d8-8dfe-656b76b65af8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.606838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.607659 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.607953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.608237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.609496 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.610973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.612656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 30/55] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:57.618366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:36:57.622552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:36:57.641931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.642330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.643186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.643186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.643738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.domain.id=default, user.id=4118f6e846e744a8a962f12a116b5928) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:36:57.644056 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.645568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184a0e0-ea78-438c-8da9-2aef1fac614b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.652701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 26/56] 10.222.0.21 () {56 vars in 1322 bytes} [Wed May 25 13:36:57 2022] GET /identity/v3/role_assignments?user.id=4118f6e846e744a8a962f12a116b5928&scope.domain.id=default&role.id=698416d22e5b4c278ff780c5eef36ede => generated 530 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:36:57.742516 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869126-d849-4446-a019-2901b15e9681 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:36:57.783837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:36:57.784058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:36:57.784209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:36:57.784494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:36:57.785269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:36:57.786799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c71ac6-bd35-4289-abbf-cfce2e001319 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:36:57.817557 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 31/57] 10.222.0.21 () {60 vars in 1140 bytes} [Wed May 25 13:36:57 2022] POST /identity/v3/projects => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:37:00.081267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004959ea-cf37-4c34-8074-bed732095466 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:00.081791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004959ea-cf37-4c34-8074-bed732095466 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:00.081878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004959ea-cf37-4c34-8074-bed732095466 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:00.083228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 27/58] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:00.089621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:00.090062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:00.090170 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:00.107631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:00.108880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:00.158439 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:00.177201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 32/59] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:00.201307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004959ea-cf37-4c34-8074-bed732095466 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:00.298342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:00.334358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004959ea-cf37-4c34-8074-bed732095466 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:00.373221 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:00.374023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:00.374309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:00.374590 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:00.375781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:00.377317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2197dfb-80bd-4e63-9749-66b6c4858629 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:00.378947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 28/60] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:00.385069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:00.487250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e35b-4ee5-4832-9cf7-e022271cbecd None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:00.521253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:00.521936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:00.522231 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:00.522518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:00.523239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:00.525528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31860bd2-6c11-4b49-92ae-efb738158f82 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:00.552169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 33/61] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:00 2022] POST /identity/v3/users => generated 275 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:03.991223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47341d03-df76-4328-9c36-e7581be4c544 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:03.992256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47341d03-df76-4328-9c36-e7581be4c544 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:03.993403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47341d03-df76-4328-9c36-e7581be4c544 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:03.995114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 29/62] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:04.001227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374d64-bfb3-4146-869e-284645098b07 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.001731 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374d64-bfb3-4146-869e-284645098b07 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.001802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0374d64-bfb3-4146-869e-284645098b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.026101 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e0374d64-bfb3-4146-869e-284645098b07 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:04.027322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e0374d64-bfb3-4146-869e-284645098b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:04.081125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:04.131366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 34/63] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:04.168466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47341d03-df76-4328-9c36-e7581be4c544 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.257797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89ee3e-cf28-4173-bca9-5c10722df09e None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.259141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89ee3e-cf28-4173-bca9-5c10722df09e None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.259677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89ee3e-cf28-4173-bca9-5c10722df09e None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.261401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 35/64] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:04.267584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69424e-3ec8-4467-a06b-da65c6cf8b90 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.268030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69424e-3ec8-4467-a06b-da65c6cf8b90 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.268582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69424e-3ec8-4467-a06b-da65c6cf8b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.298919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-ae69424e-3ec8-4467-a06b-da65c6cf8b90 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:04.300458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-ae69424e-3ec8-4467-a06b-da65c6cf8b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:04.330660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47341d03-df76-4328-9c36-e7581be4c544 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:04.403326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:04.404578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.406110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.407416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.426944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:04.428232 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:04.432780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 36/65] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:04.434332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80881-6440-49aa-84cd-764450fba15b None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.434805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80881-6440-49aa-84cd-764450fba15b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.435227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c80881-6440-49aa-84cd-764450fba15b None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.437502 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 37/66] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:04.439062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:04.439921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:04.440734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:04.443385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 30/67] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:04.451746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.451746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.451746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.452057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.467941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:04.469617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:04.543727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 38/68] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:04.545049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.575466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb299fdf-9624-43c8-ae55-41d35be73fff None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:04.589920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:04.620352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:04.621464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.622138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.622651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.624264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:04.624891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:04.627204 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de4838-2978-4822-9492-2494568ffbd8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:04.637447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 31/69] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:04.638955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efc2c6-4812-4a0c-b504-273a9ac2b86b None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.639253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efc2c6-4812-4a0c-b504-273a9ac2b86b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.639536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efc2c6-4812-4a0c-b504-273a9ac2b86b None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.640959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 32/70] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:04.645843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efc2c6-4812-4a0c-b504-273a9ac2b86b None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.681699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b0ca5-8abd-4016-8e4b-7e5eb8521d00 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:04.715233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:04.732198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:04.733125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.733415 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.733696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.747604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:04.755375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:04.756221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 39/71] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:04.757510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.767438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efc2c6-4812-4a0c-b504-273a9ac2b86b None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:04.801151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:04.801887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.802181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.802459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.813984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-4f34064b-87ce-483c-a8ce-d85a02368124 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:04.822375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:04.823382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:04.824149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 33/72] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:04.824940 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff917-2c5d-4c9b-94c4-2c8b27846994 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.825191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff917-2c5d-4c9b-94c4-2c8b27846994 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.825425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff917-2c5d-4c9b-94c4-2c8b27846994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.842682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-8a2ff917-2c5d-4c9b-94c4-2c8b27846994 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:04.843895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-8a2ff917-2c5d-4c9b-94c4-2c8b27846994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:04.852473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:04.862392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9df8-ae04-4682-92b3-23550199bedb None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:04.897705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:04.898377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.898705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.915434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 34/73] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:04.916046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:04.916848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba75-0fe7-4549-b6bc-cab5c0c29e92 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:04.917147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba75-0fe7-4549-b6bc-cab5c0c29e92 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:04.917429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba75-0fe7-4549-b6bc-cab5c0c29e92 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:04.917660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:04.920868 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 35/74] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:04.921022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 40/75] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:04.921095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ba75-0fe7-4549-b6bc-cab5c0c29e92 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.921161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:04.977563 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.041498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ba75-0fe7-4549-b6bc-cab5c0c29e92 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.042694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75296e37-f05a-435e-8b33-97906cf2afea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.079766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.080358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.080682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.080969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.081737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:05.082050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.083484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75046f63-22e5-4164-b696-2c4eaaf34460 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.087742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.088521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.089159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.089449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.090232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:05.090550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.091988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.092782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 36/76] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.094105 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.094399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.094681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.101161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 41/77] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.102538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.114750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:05.115586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.115968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:05.189022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 37/78] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:05.190194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.209458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec1c4b-d0dd-4243-9562-e24e0d3f33e2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.251147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.259746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.260420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.260743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.261029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.316075 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6503d-ce08-4f7e-8221-75e5a3f51807 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.331981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.333457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.345526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.356414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.359371 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.359658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.359938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.370764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:05.379201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:05.380065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:05.380892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 38/79] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:05.383543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.443248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 42/80] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:05.444089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.491060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37d733-ead9-4e1a-a0e4-bd49add0db52 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.492819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.529574 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.530468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.530759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.531043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.532268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.533794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.535432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 39/81] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.536201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.557180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada03e55-48b8-40ec-bba3-b3415828325a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.567919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.593334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.594143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.594430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.594713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.595934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.597470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.599127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 43/82] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.599909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.649985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b49b9-c7c0-45ed-a00c-10f8f2f72f82 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.686102 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.686904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.687185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.687467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.688672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.690146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.691771 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 40/83] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.692543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.706911 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b4183-4482-4d0c-9f55-3a2e5076c444 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.733026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.742159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.742872 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.743184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.743621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.744379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:05.744727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.746132 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.754660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 44/84] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.755309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.797299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.798433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c45d5e-bfe8-4711-9240-d2615704c3c1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.832885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.833777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.834108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.834413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.835618 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.837721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.839943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 41/85] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:05.840548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:05.857526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a71f1-d0a5-4b8e-b7ba-118f74cd3e94 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.892216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.892925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.893212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.893485 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:05.941789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a51fd4-7dd4-464a-a20d-056d1a47f94f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:05.949716 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:05.963859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:05.965865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:05.979772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:05.980450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:05.980776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:05.981064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.010906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.053511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.055535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.071317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 45/86] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:06.072164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.159567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 42/87] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:06.160436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.169131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.177096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0004cc7-e6b2-49a9-bd05-7353d4c36c47 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.211983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.212801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.213094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.213440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.214434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.215883 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.225713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.237403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 46/88] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:06 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:06.238241 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.268167 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f93611-8432-464e-bba3-f1b7d0af4116 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.318247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.319599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.320757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.321035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.322526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.324809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.331154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 43/89] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:06.331154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.373455 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ddebe-3268-45af-899a-792e1ee46d84 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.412221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.412942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.413237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.413557 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.414479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:06.414803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.416303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.419889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.427301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 47/90] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:06.427301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.451119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe382e6-0b50-4de2-9185-cb3c1b256b91 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.509292 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.510182 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.510477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.510763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.512274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.513950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.516261 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 44/91] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:06.517015 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.533002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.610392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfaf24-80cd-49cf-ac7f-e211721b3762 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.668947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae0a7bd-2b03-442b-a3d7-30e7c6478475 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.669355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.670678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.670983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.672244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.673590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.675102 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.677487 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 48/92] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:06.678296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.712045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.712781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.713080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.713366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.733589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.805658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff863831-cda7-4409-bc47-9177fc8e2eaa None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:06.809609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.811206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.841333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:06.852914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:06.853612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:06.853907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:06.854196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:06.855050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=4118f6e846e744a8a962f12a116b5928) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:06.855375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:06.856902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:06.871682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 49/93] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/role_assignments?user.id=4118f6e846e744a8a962f12a116b5928&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:06.874387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:06.986599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 45/94] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:06.991708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:07.078465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:07.093256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3899-094d-40b6-8ff4-4d1e42c8add4 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:07.171234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:07.171908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:07.172263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:07.172550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:07.173499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:07.173825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:07.175393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:07.186101 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 50/95] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=13f5475a26344e9e816475b8539030d3 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:07.205488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157464a-fb33-4c0d-87d2-f9b10ac71127 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:07.275237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:07.276648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:07.277240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:07.277742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] PATH_INFO: `/v3/projects/76e5ca9ce31f45bf890378d633727674` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:07.285652 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] RBAC: Authorizing `identity:get_project(project_id=76e5ca9ce31f45bf890378d633727674)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:07.287573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39a2eb-061a-45f9-9145-aba4056bf291 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:07.290278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 46/96] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:06 2022] GET /identity/v3/projects/76e5ca9ce31f45bf890378d633727674 => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:07.297721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:07.397230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:07.458330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e0e4e-b4f9-4118-b511-4e1fe438cfdc None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:07.501774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:07.502458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:07.502749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:07.503036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:07.503884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=76e5ca9ce31f45bf890378d633727674, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:07.504206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:07.505713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fdfb5-671f-498e-8e18-84c71991457a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:07.516316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 51/97] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:07 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=76e5ca9ce31f45bf890378d633727674&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:09.419978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cdf5-77d0-4463-87dd-6c2e9107c719 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.421036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cdf5-77d0-4463-87dd-6c2e9107c719 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.421533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830cdf5-77d0-4463-87dd-6c2e9107c719 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.423160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 47/98] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:09.428936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.431126 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.431622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.452519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:09.454184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:09.475052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d86db-709c-4bc1-be3a-4282083c8a8b None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.475373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d86db-709c-4bc1-be3a-4282083c8a8b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.475701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d86db-709c-4bc1-be3a-4282083c8a8b None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.477705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 48/99] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:09.494725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683685-6a47-4251-a385-2dc8135ba705 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.495023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683685-6a47-4251-a385-2dc8135ba705 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.495306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea683685-6a47-4251-a385-2dc8135ba705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.523080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-ea683685-6a47-4251-a385-2dc8135ba705 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:09.525045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-ea683685-6a47-4251-a385-2dc8135ba705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:09.639782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 52/100] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:09.686237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:09.687278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 49/101] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 25 13:37:09.726660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25f7-7aad-4d92-a3bd-2e70e07af8d0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.727464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25f7-7aad-4d92-a3bd-2e70e07af8d0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.727977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25f7-7aad-4d92-a3bd-2e70e07af8d0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.729867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 50/102] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:09.730818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d25f7-7aad-4d92-a3bd-2e70e07af8d0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:09.768164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:09.817596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:09.860847 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc128ca5-9b41-4369-b714-680ab13cdd7b None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:09.926585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:09.927419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.927718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.928003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:09.934104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d25f7-7aad-4d92-a3bd-2e70e07af8d0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:09.956500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e0746294-1f90-41ee-90b4-3b8d79036326 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:09.964745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:09.965679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:09.966427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 53/103] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:09.978298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:09.978667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:09.978959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.005742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.006561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.006851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.007136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.009429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:10.010920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:10.012599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 51/104] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:10.013405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.017843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:10.019112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:10.150961 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 54/105] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:10.151977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.221850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:10.253189 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748cde6-ae66-4aca-a656-92bf52f78d3c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.264078 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:10.291707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.292400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.292723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.293008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.293772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:10.294089 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:10.295712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:10.305733 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 52/106] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:10.306830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.370858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c236f92-6c7e-444e-8ac0-4e924cbdc004 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.421876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c658a9c-fd94-48ec-8486-d54b02a05d47 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.454649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.457344 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.458365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.459265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.461883 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:10.464209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:10.468651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.469672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.471415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.471845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.493449 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:10.504013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:10.505185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:10.506203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:10.508399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 53/107] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:10.509524 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.522731 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 55/108] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:37:10 2022] POST /identity/v3/projects => generated 302 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:10.529138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.562522 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:10.655899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97606dbe-2479-44a5-990b-0aae4c9b1174 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.663343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:10.678434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0e526-32f7-427d-8509-6d36678d0899 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.694621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.695955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.696252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.696508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] PATH_INFO: `/v3/users/b396acc58c5d4ab0a79708bf94e74b49` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.720029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.721162 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.721700 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.722195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.723189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:10.723715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:10.725394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] RBAC: Authorizing `identity:get_user(user_id=b396acc58c5d4ab0a79708bf94e74b49)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:10.725697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfc4531-c9a1-477c-a05b-314e72be7242 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:10.727561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8a0f-7583-4497-8af3-aec5ffc70004 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:10.729467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 54/109] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/users/b396acc58c5d4ab0a79708bf94e74b49 => generated 289 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:10.730828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80f132-fd5a-4296-95a3-6d2c74d40e75 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.731120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80f132-fd5a-4296-95a3-6d2c74d40e75 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.731402 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80f132-fd5a-4296-95a3-6d2c74d40e75 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.732838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 55/110] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:10.741739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80f132-fd5a-4296-95a3-6d2c74d40e75 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.744038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 56/111] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:10.745825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.746332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.746823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.786803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:10.789355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:10.876691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 57/112] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:10.878627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:10.908166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:10.923663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80f132-fd5a-4296-95a3-6d2c74d40e75 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:10.962148 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:10.962815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:10.963353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:10.963642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:10.982697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.000000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371eb2aa-e0dd-40cb-886c-53a786ea09bd None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.036123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.037062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.037354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.037642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.039408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.040928 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361ff93-1e8d-4958-8869-018ed71ad608 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.042617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 58/113] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:11.043859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946872-0c38-4c11-8651-7db7e7528990 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.044150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946872-0c38-4c11-8651-7db7e7528990 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.044433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946872-0c38-4c11-8651-7db7e7528990 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.045868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 59/114] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:11.046809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc34-95b0-4fad-923c-775cbab580a1 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.047088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc34-95b0-4fad-923c-775cbab580a1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.047373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8fc34-95b0-4fad-923c-775cbab580a1 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.052544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 60/115] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:11.052544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8fc34-95b0-4fad-923c-775cbab580a1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.055373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.057150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a474da-d493-40f9-aaab-2545058ee7d9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.148697 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.161061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8fc34-95b0-4fad-923c-775cbab580a1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.167995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 56/116] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:11.167995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c8378-379b-47c9-9174-cb27e5559e13 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.167995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c8378-379b-47c9-9174-cb27e5559e13 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.167995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c8378-379b-47c9-9174-cb27e5559e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.181565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-4e4c8378-379b-47c9-9174-cb27e5559e13 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:11.182772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-4e4c8378-379b-47c9-9174-cb27e5559e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:11.203221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.204149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.204440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.204750 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.217361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:11.225664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:11.227147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:11.227906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 61/117] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:11.233567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.247133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.268842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 57/118] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:11.270380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.270671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.270951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.292476 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:11.293959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:11.358722 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfd960-d249-4061-bbcc-07d53f0b22de None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.364789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 58/119] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:11.369514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.401023 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.401925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.402199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.402481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.403768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.405189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.406903 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 62/120] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:11.407683 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.432743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.485993 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa548a6e-ccf9-4aa4-b9df-eebd6a443d78 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.503222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.524034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.524842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.525126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.525405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.526612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.528094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.528792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4ef88-35fa-41aa-9034-784783b5252e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.529837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 59/121] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:11.530609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.574058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.575049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.575541 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.576099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.576975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:11.577296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.578822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.593645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 63/122] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:11.593645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.647006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da576e-e00c-4465-b92d-ed2916d6c089 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.669554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.689075 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.690157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.690454 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.690735 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.699972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6ce76-9acf-4c21-8c33-d66c4ad2380c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.701507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:11.709262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:11.710281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 60/123] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:11.716667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.747077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.747897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.748180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.748522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.759446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:11.767582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:11.768505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:11.769343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 64/124] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:11.769984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.773717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:11.830772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9f9b-7ccb-4587-9a8a-f8031b873420 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.869607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.870520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.870814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.871098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/users/fc1945641a034317ac1e53f84a83c2a2/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.873362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, user_id=fc1945641a034317ac1e53f84a83c2a2, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.875038 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.883597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2af24-633e-46ca-952f-76d11523d5af None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:11.900594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 61/125] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:11 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/users/fc1945641a034317ac1e53f84a83c2a2/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:11.900594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.930790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:11.931481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:11.931767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:11.932055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:11.932993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=b396acc58c5d4ab0a79708bf94e74b49) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:11.933322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:11.934839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:11.941924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 65/126] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/role_assignments?user.id=b396acc58c5d4ab0a79708bf94e74b49&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:11.949729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:11.961349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:12.060416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d344e3-80da-484c-8f4a-c0ed2171becd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.109179 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.110461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.110998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.111487 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.113134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.115806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.118260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 62/127] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.119530 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.155251 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:12.207689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21baa234-42b8-4c3c-95dc-165ef18b9faf None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.250718 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb93e8-8f08-411e-8744-85ac7e621a7a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.254082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:12.274154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.274838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.275122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.275398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.276157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:12.276467 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.277898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.292666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 66/128] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.293935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.306999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.307832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.308243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.308655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.309540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:12.309953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.311473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.322433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 63/129] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.323266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.494499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749610a-5726-4a2c-8d87-65e9ccb9fdea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.527185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726c8eb-8674-4998-8e13-ab30c80fba23 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.551240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:12.570099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.572673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.572673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.573375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] PATH_INFO: `/v3/projects/76e5ca9ce31f45bf890378d633727674` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.575073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] RBAC: Authorizing `identity:get_project(project_id=76e5ca9ce31f45bf890378d633727674)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.576596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.578358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 67/130] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/projects/76e5ca9ce31f45bf890378d633727674 => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.579145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.581531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.582435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.582859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.583044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.586810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.588174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.594131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 64/131] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.594131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.636106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:12.709342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920caecc-1e9c-4705-bc62-b92f687f3245 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.754496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.755324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.755614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.755896 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] PATH_INFO: `/v3/projects/76e5ca9ce31f45bf890378d633727674/users/fc1945641a034317ac1e53f84a83c2a2/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.760479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] RBAC: Authorizing `identity:create_grant(project_id=76e5ca9ce31f45bf890378d633727674, user_id=fc1945641a034317ac1e53f84a83c2a2, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.761989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.787862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 65/132] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:12 2022] PUT /identity/v3/projects/76e5ca9ce31f45bf890378d633727674/users/fc1945641a034317ac1e53f84a83c2a2/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:12.788967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.833266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990662f7-4749-42f9-be8e-32aecb026d3e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:12.920051 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:12.921007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:12.921296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:12.921588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:12.926959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:12.927146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:12.929131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 68/133] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:12.936058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:12.992421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:13.012017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8347-6494-4884-9cc8-65b68b516370 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.090688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.091765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.092057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.092338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.097513 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.099894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.102412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 66/134] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:13.107979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:13.109148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:13.180990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cae3b-b6d7-4cff-a58f-67a8c1e58fca None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.219131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.220062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.220358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.220667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.223907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.225505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9318f-204d-44f7-b22f-7b2930813c13 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.227323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 69/135] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:12 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:13.228761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f36ff2-e589-4f5c-86ac-00809fefda5f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.229056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f36ff2-e589-4f5c-86ac-00809fefda5f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.229337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f36ff2-e589-4f5c-86ac-00809fefda5f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.230743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 70/136] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:13.233020 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f36ff2-e589-4f5c-86ac-00809fefda5f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:13.256439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c805-3299-4586-a3d1-7545ada2a9ce None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.308992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.309873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.310223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.310511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/users/4118f6e846e744a8a962f12a116b5928/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.313089 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, user_id=4118f6e846e744a8a962f12a116b5928, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.315181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46a78f-a178-486b-84e8-a700ef9b91a7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.337528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 67/137] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:13 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/users/4118f6e846e744a8a962f12a116b5928/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:37:13.339989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.341790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.342674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.381708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:13.383222 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:13.415614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f36ff2-e589-4f5c-86ac-00809fefda5f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.418226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:13.481773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:13.493281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.494241 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.494530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.494810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/users/fc1945641a034317ac1e53f84a83c2a2/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.498370 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, user_id=fc1945641a034317ac1e53f84a83c2a2, role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.501548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.523565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 71/138] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:13 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/users/fc1945641a034317ac1e53f84a83c2a2/roles/13f5475a26344e9e816475b8539030d3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:37:13.562266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 68/139] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:13.597033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:13.722391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e767f-2bd0-4d65-a265-2db7122d6b3e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.761225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.762446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.762840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.763224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.764597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.766103 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1d74e-896b-421c-8da1-bcbd1ef12e35 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.767873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 72/140] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:13.774354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:13.893515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d601dc-fae3-4d4b-8fda-1fd88845e00b None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:13.930888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:13.931646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:13.931938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:13.932225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:13.933147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:13.934636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cdd42-df27-4dea-8804-b587628db31d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:13.965345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 69/141] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:13 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:15.122663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9898d4-62a4-4442-bdc5-1d79371a8b60 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.123187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9898d4-62a4-4442-bdc5-1d79371a8b60 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.123270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9898d4-62a4-4442-bdc5-1d79371a8b60 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.124790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 73/142] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:15.130346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63accc94-aa8e-4a50-b8cb-b35bfb682f8c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.130813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63accc94-aa8e-4a50-b8cb-b35bfb682f8c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.130901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63accc94-aa8e-4a50-b8cb-b35bfb682f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.158005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-63accc94-aa8e-4a50-b8cb-b35bfb682f8c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:15.159360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-63accc94-aa8e-4a50-b8cb-b35bfb682f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:15.161657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:15.263570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 70/143] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:15.282783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1345-bf70-4d6a-9325-704728fe6def None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.283081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1345-bf70-4d6a-9325-704728fe6def None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.283555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b1345-bf70-4d6a-9325-704728fe6def None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.285121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 74/144] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:15.297437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.297734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.298023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.321609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:15.323219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:15.339856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b1345-bf70-4d6a-9325-704728fe6def None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:15.441915 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 71/145] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:15.464132 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:15.477543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:15.550954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b1345-bf70-4d6a-9325-704728fe6def None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:15.605318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:15.606259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.606630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.606985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.608698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:15.629294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:15.643438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:15.644352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:15.645183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 75/146] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:15.660703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:15.698943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190a7d8-aed9-49c8-adb5-4d4bad1eb65e None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:15.775739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:15.776600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.776981 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.777264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.800240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:15.805721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a73a5-1752-49fa-a918-9bbe8752fea8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:15.809248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:15.811595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:15.813546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 72/147] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:15.822070 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:15.832353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:15.864421 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:15.868072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:15.868364 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:15.870352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:15.871429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:15.871739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:15.875314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:15.890831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 76/148] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:15.896828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.030252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7dcb3-5a1b-4eaf-b393-12103c40b999 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.052506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f68c10-dc2e-4a51-b6c7-c3a9515a490f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.071613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.072507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.072841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.073121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.074452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.076892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.078755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 77/149] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:15 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:16.085175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.108016 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.126840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.127624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.127916 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.128197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.128990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:16.129302 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.130725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.145127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.147777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 73/150] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:16.153728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.211303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc081c3f-05ca-4ac9-843c-1d8063f27ca6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.260729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.261592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.261885 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.262900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.354099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.355862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.363208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467de58e-1817-4d40-bee9-c84819359058 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.417076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.430498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.431765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.432281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.435548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] PATH_INFO: `/v3/users/b396acc58c5d4ab0a79708bf94e74b49` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.437145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] RBAC: Authorizing `identity:get_user(user_id=b396acc58c5d4ab0a79708bf94e74b49)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.438810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.440897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 74/151] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/users/b396acc58c5d4ab0a79708bf94e74b49 => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:16.447114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.483899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 78/152] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:16.485527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.568492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.618693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084e282-e2ea-49fa-b796-bddbba910e96 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.670061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.671237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.671734 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.672166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.677348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.679342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072af40-af17-4d48-932d-eff85d9bacee None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.681425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 79/153] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:16.682894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd7bde-4227-4c1e-9c7c-77fd2d6b4ae3 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.683345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd7bde-4227-4c1e-9c7c-77fd2d6b4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.683761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd7bde-4227-4c1e-9c7c-77fd2d6b4ae3 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.685258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 80/154] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:16.689344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c787447b-f949-4505-905b-cdaca934f645 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.690752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.691048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.691326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.721676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:16.723154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:16.748919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.749322 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.750234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.750578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.750985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.752392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.754063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.755883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 75/155] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:16.756950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.850385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 81/156] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:16.851631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:16.930091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00b3c-01b6-4a02-86ec-0aacb26e13f4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:16.968594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:16.985275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:16.986211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:16.986658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:16.987234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:16.988250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:16.988819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:16.990403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:16.997516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 76/157] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:16.998522 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:17.112594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d8dce-60e5-4934-b8de-0361a57b920d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:17.155989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff908b-2746-479f-92af-a7a75b35dce1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:17.217552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:17.218616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.219054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.219497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.235723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:17.246538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:17.248064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.248506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:17.248906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:17.250062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:17.251022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:17.251762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/users/b396acc58c5d4ab0a79708bf94e74b49/roles/865913f1b8fb4d0db423c6c32d8f08fa` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.253460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 77/158] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:17.258286 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, user_id=b396acc58c5d4ab0a79708bf94e74b49, role_id=865913f1b8fb4d0db423c6c32d8f08fa)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:17.259865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:17.260616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:17.294974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 82/159] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:16 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/users/b396acc58c5d4ab0a79708bf94e74b49/roles/865913f1b8fb4d0db423c6c32d8f08fa => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:17.330659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:17.410797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b1a39-96f1-4a99-b442-b6db51ad1b73 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:17.461615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:17.462944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.463451 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.463949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.464945 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:17.465467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:17.467090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c773-5a92-4612-935a-48797fd14e78 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:17.481867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 78/160] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:17.487239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:17.503789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:17.564123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47bd8b-5d20-4969-a171-9815ee79fca7 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.564467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47bd8b-5d20-4969-a171-9815ee79fca7 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.564717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47bd8b-5d20-4969-a171-9815ee79fca7 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.566383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 79/161] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:17.572746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379618d-d89c-4cb6-8372-46593f534bce None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.573342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379618d-d89c-4cb6-8372-46593f534bce None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.573823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379618d-d89c-4cb6-8372-46593f534bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.627831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e379618d-d89c-4cb6-8372-46593f534bce None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:17.629440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e379618d-d89c-4cb6-8372-46593f534bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:17.669211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc9a94-91a5-44ad-a0e5-a12e5f7759a2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:17.749267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:17.750596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.751174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.751604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.753309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:17.755478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001ff5e-5e75-484c-980e-01563dc2fea2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:17.757807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 83/162] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:17 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:17.759283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131182b-aea4-4f14-a78c-72b16d6b692b None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.759923 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131182b-aea4-4f14-a78c-72b16d6b692b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.760367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131182b-aea4-4f14-a78c-72b16d6b692b None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.766144 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 84/163] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:17.767166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131182b-aea4-4f14-a78c-72b16d6b692b None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:17.794408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 80/164] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:17.795781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:17.796076 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:17.796358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:17.825701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:17.826985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:17.891362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:17.943109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 81/165] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:17.945087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:17.986606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.033389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131182b-aea4-4f14-a78c-72b16d6b692b None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.066339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f596cc0-2bc9-4867-8206-23ede590a3ae None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.096914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.097812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.098198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.098564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.109911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.110795 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.111118 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.111526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.130069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:18.143552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:18.144477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:18.145544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 82/166] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:18.146199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.197866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.219716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:18.226540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:18.348021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.393864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 85/167] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:18.394640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.414809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d0d01-ea2c-43d1-bf15-bd7ba46a4465 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.505218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.506032 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.506441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.506775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.530297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:18.543671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:18.544649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:18.545445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:18.546742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 83/168] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:18.547460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.645772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237f49c-48f7-4807-9a7f-10e5a4508cc5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.701806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.713412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.714145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.714529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.714808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.715679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:18.715991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:18.717472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:18.721234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84147538-2cd9-4e51-bf8d-1ebdb5840534 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.727819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 86/169] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:18.732751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.758119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.759092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.759383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.759657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] PATH_INFO: `/v3/projects/76e5ca9ce31f45bf890378d633727674` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.761239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] RBAC: Authorizing `identity:get_project(project_id=76e5ca9ce31f45bf890378d633727674)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:18.762780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:18.764536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 84/170] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/projects/76e5ca9ce31f45bf890378d633727674 => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:18.765377 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.806078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.880855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00478b44-98c0-495a-b957-380ac51ade8e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:18.918958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:18.924587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:18.924587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:18.924587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] PATH_INFO: `/v3/users/fc1945641a034317ac1e53f84a83c2a2` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:18.924587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] RBAC: Authorizing `identity:get_user(user_id=fc1945641a034317ac1e53f84a83c2a2)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:18.924587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:18.924984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 85/171] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/users/fc1945641a034317ac1e53f84a83c2a2 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:18.925960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:18.963326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:18.978524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7620bd8-87a4-4d90-9eb6-bba9964fd7ed None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.039826 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c833-3284-4c10-830c-4ec05d7a2a3d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.078436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.079442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.079860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.081557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=76e5ca9ce31f45bf890378d633727674, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:19.081971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.083970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.091592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 86/172] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=76e5ca9ce31f45bf890378d633727674&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:19.092646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.098381 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.099341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.099783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.100794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.101736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:19.102208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.103834 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bf92f-3953-42ee-84ec-7f6f8d64a047 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.130865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 87/173] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:19.132822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c113f-1ed9-42bd-9f48-b79e1a0e7019 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.133314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c113f-1ed9-42bd-9f48-b79e1a0e7019 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.133730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c113f-1ed9-42bd-9f48-b79e1a0e7019 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.135322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 88/174] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:19.140908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.141381 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.141818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.184882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:19.186333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:19.232969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.244289 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99371a6-86fb-42d6-9b47-307828c640f6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.287290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.288497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.289236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.289723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.369154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.371182 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b0993-cf9f-433a-90f0-4d4694a220ea None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.402672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 89/175] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:19.403569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.449783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.486526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.498647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 87/176] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:19.500407 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71809b6f-0957-4bde-a84b-387080307989 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.500920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71809b6f-0957-4bde-a84b-387080307989 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.501396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71809b6f-0957-4bde-a84b-387080307989 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.502899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 88/177] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:19.503964 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71809b6f-0957-4bde-a84b-387080307989 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.541241 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124bca1-16ac-4615-bb52-5b5b0b15380f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.597383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.599736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.599736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.599736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] PATH_INFO: `/v3/users/4118f6e846e744a8a962f12a116b5928` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.603213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=4118f6e846e744a8a962f12a116b5928)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.604721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.606486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 90/178] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/users/4118f6e846e744a8a962f12a116b5928 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:19.607174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.645304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71809b6f-0957-4bde-a84b-387080307989 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.682402 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.683321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.683605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.683879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.694448 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5a8f6df5-c0f4-4c2f-a66d-1d56347a42c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:19.702085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:19.702962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:19.703751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 89/179] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:19.704427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c34fd2-d4bf-4689-aee7-ce9b3d7bb8c4 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.704714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c34fd2-d4bf-4689-aee7-ce9b3d7bb8c4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.704969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c34fd2-d4bf-4689-aee7-ce9b3d7bb8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.724523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-42c34fd2-d4bf-4689-aee7-ce9b3d7bb8c4 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:19.726375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-42c34fd2-d4bf-4689-aee7-ce9b3d7bb8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:19.731493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.735766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315e0e-dcf3-44b4-abc8-be848706a9f8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.771589 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.773083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.773478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.773757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.774941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.776382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.778151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 91/180] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:19.778915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.784873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.808112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 90/181] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:19.809753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619721f-a817-47cd-a0a3-dfbec1d2101f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.810188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619721f-a817-47cd-a0a3-dfbec1d2101f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.810740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619721f-a817-47cd-a0a3-dfbec1d2101f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.812291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 91/182] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:19.813154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619721f-a817-47cd-a0a3-dfbec1d2101f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:19.891980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d978d-895e-48a8-a9ca-e548b8b29804 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.931246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.931608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619721f-a817-47cd-a0a3-dfbec1d2101f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:19.931912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.932202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.932486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.969892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:19.970612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:19.970888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:19.971187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:19.971927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:19.972352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:19.975021 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:19.979551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:19.985691 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 92/183] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:19.990918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.018969 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.020365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b1092-ca66-43c2-a635-019f16288585 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.040252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.095438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d9207-11d0-4a23-a0aa-fbb88153bc6e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.129243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.129910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.130198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.135380 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 92/184] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:20.135526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.135526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=fc1945641a034317ac1e53f84a83c2a2) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:20.135526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.135741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.135741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.135741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.135947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.141417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 93/185] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:19 2022] GET /identity/v3/role_assignments?user.id=fc1945641a034317ac1e53f84a83c2a2&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=13f5475a26344e9e816475b8539030d3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.142096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.166183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:20.167999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:20.257867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.272155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 93/186] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:20.273685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.282786 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352564a-3bec-4cf7-9d2c-20b77a136cf4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.306616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.345325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.346776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.347069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.347353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.364859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:20.373151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:20.374028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 94/187] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:20.375157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.436462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d086c7-89f2-42b8-b005-7d254b688c02 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.480655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.480655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.480655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.481003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.496483 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.498088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.499915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 94/188] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.500767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.513343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59d41-22f8-409c-b185-aa52588cf854 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.556257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.557444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.557724 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.558976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.560509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.566762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 95/189] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.566762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.578787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.599538 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.652739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884d15b-d7b3-4d9f-83ef-fdf5477bc8d3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.691905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.692143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072f516-4c49-4646-89fc-b1568399952a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.692741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.693033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.693314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.694809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.696274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.702198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 95/190] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.702198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.731317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.731990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.732279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.732570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.733340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:20.733657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.735048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.743894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 96/191] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.744797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.800654 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb805e3-64b8-4934-8ddd-16f432e815ad None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.824175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.836035 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.836719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.837005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.837535 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.840819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:20.844694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900689d-399e-4e20-80bc-fba919b1ab8e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:20.878729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:20.879383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:20.879665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:20.879939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:20.880796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=4118f6e846e744a8a962f12a116b5928) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:20.881144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.882591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:20.888349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 97/192] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/role_assignments?user.id=4118f6e846e744a8a962f12a116b5928&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:20.889321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:20.915987 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:20.917938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6611d0-8bed-46b3-b722-823ba237dece None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.020941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aafa00-786e-437a-a751-11cb3342f2b4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.045541 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 96/193] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:21.047233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dce3a-fe7a-4753-a935-14dec8cc7b0d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.047524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dce3a-fe7a-4753-a935-14dec8cc7b0d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.048021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7dce3a-fe7a-4753-a935-14dec8cc7b0d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.054106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 97/194] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:21.054653 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dce3a-fe7a-4753-a935-14dec8cc7b0d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.063656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.064341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.064666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.064949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.065673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.067151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.083374 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.096708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 98/195] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:20 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:21.096708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.120080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.213417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7dce3a-fe7a-4753-a935-14dec8cc7b0d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.244909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80ab8-5d71-494d-85d6-9ea08199fdb5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.253124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.254054 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.254343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.254627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.256632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.258360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2c00-4094-4f39-8e62-29a92643f296 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.260198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 98/196] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:21.261764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.262062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.262326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.284978 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:21.286537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:21.290695 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.291599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.292064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.292482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.294008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.296124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.298152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 99/197] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:21.299154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.387756 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 99/198] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:21.389007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.415476 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.415765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.440156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5d075-9385-4d30-b66b-29ef909a8217 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.489784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.491113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.491632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.492103 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.541127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014430ce-c4dc-4acd-8b68-c669df7da812 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.575448 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.577343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.578991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.579814 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.580254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.580529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.581400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=4a6862654bb14ea385a578ed7cf8788b) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:21.581864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.583457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.591272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 100/199] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/role_assignments?user.id=4a6862654bb14ea385a578ed7cf8788b&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:21.592388 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.755915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.771369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:21.791059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9d76d-a12f-434b-b265-ff3a324c0ef4 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.805693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 100/200] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:21.811844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.846773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.848487 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.849207 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.849669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.863320 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:21.874844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:21.875946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:21.876959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:21.879839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 101/201] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:21.889621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:21.937426 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c4bd1-c27e-4d69-96b9-aec2aa6a507c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:21.974912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:21.975796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:21.976183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:21.976575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:21.977920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:21.979767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:21.986315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 101/202] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:21.987412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:22.094295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:22.099411 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e14ab-53b6-494a-9b1e-f56029b61042 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:22.145009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:22.145694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:22.145987 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:22.146271 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:22.147079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:22.147392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:22.148836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:22.158019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 102/203] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:22.164160 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:22.180932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:22.184961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c09b6-8405-4f00-b99f-b53abf3bb7a0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:22.256457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:22.257153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:22.257502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:22.257882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:22.258789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=4a6862654bb14ea385a578ed7cf8788b) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:22.259112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:22.260592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:22.267316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 102/204] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:21 2022] GET /identity/v3/role_assignments?user.id=4a6862654bb14ea385a578ed7cf8788b&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=13f5475a26344e9e816475b8539030d3 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:22.381224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c8ed0-f144-44e5-98e8-4b928fc05328 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:22.425587 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:22.426479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:22.426763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:22.427041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] PATH_INFO: `/v3/users/b396acc58c5d4ab0a79708bf94e74b49` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:22.428851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] RBAC: Authorizing `identity:get_user(user_id=b396acc58c5d4ab0a79708bf94e74b49)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:22.430340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:22.432247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 103/205] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:22 2022] GET /identity/v3/users/b396acc58c5d4ab0a79708bf94e74b49 => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:22.438003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:22.565913 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478be5-4a92-4071-bcea-991df4a64a7c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:22.585778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:22.606426 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:22.607093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:22.607388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:22.607672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:22.680912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:22.682405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:22.791592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 103/206] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:22.797330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:22.876939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:22.903707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32984bf8-1734-40ab-b6c4-15a0d580b765 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:22.938817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:22.939615 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:22.939906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:22.940187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:22.941464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:22.942993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c24d56-18ff-4a61-abcf-efa91d1761d6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:22.945417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 104/207] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:22 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:22.951361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:22.972968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:23.074310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3f04e-f1f9-40ec-bafb-f82d81934e49 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:23.111464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:23.112134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:23.112422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:23.112789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:23.113818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, scope.project.id=7ab3476e508d41e482b975ae625333e3, user.id=b396acc58c5d4ab0a79708bf94e74b49) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:23.114228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:23.116613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eaf5d-ed21-4064-bdaa-1d04e508a4e3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:23.123246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 104/208] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:22 2022] GET /identity/v3/role_assignments?user.id=b396acc58c5d4ab0a79708bf94e74b49&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:23.952710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7151-3f2a-49bb-b206-d2fe0c44a4fa None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:23.953730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7151-3f2a-49bb-b206-d2fe0c44a4fa None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:23.954266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e7151-3f2a-49bb-b206-d2fe0c44a4fa None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:23.955858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 105/209] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:23.962410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839268-b5a3-427f-82ba-b0230217694a None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:23.963339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839268-b5a3-427f-82ba-b0230217694a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:23.963844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839268-b5a3-427f-82ba-b0230217694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:23.983271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-91839268-b5a3-427f-82ba-b0230217694a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:23.984892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-91839268-b5a3-427f-82ba-b0230217694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:24.054831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ff17-1f48-416d-af0c-3e05da47c212 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.055149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ff17-1f48-416d-af0c-3e05da47c212 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.055401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081ff17-1f48-416d-af0c-3e05da47c212 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.056808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 106/210] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:24.063022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81426aa-5b9a-48b1-b0a7-b039bc8100aa None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.063550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81426aa-5b9a-48b1-b0a7-b039bc8100aa None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.064130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81426aa-5b9a-48b1-b0a7-b039bc8100aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.066319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 105/211] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:24.081898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a81426aa-5b9a-48b1-b0a7-b039bc8100aa None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:24.083390 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a81426aa-5b9a-48b1-b0a7-b039bc8100aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:24.090675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91839268-b5a3-427f-82ba-b0230217694a None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.105105 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.171610 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 107/212] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:24.181911 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4ee5-3ec2-405d-9ca4-b5761396bfe1 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.182207 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4ee5-3ec2-405d-9ca4-b5761396bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.182491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e4ee5-3ec2-405d-9ca4-b5761396bfe1 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.183802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 108/213] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:24.191309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.192285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.193166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.198215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91839268-b5a3-427f-82ba-b0230217694a None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.214061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:24.215192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:24.234887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.235687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.235973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.236250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.247348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:24.255101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:24.256596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:24.257409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 106/214] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:24.258057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.294816 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 109/215] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:24.295954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.318555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.345668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.387037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d4aa-de92-4492-a1f6-cf87b543a04f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.416951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3ddfa-4155-4f03-9272-f299b6de6b27 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.432835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.433620 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.433910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.434180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.435352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:24.436947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:24.443042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 107/216] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:24.443042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.459489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.460436 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.460929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.461388 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.462651 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:24.463142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:24.464920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:24.475024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 110/217] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:24.476849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.544918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.547767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6b6de-9901-4586-b3a5-745153c6a9b2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.583856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.585054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.585526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.586025 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.589687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a079614-bb0b-4922-b76a-8c5a8f2d61af None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.601675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.605785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:24.614545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:24.616469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:24.617370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:24.621873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 108/218] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:24.622936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.638593 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.639282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.639579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.639864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.640617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:24.642088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:24.664142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 111/219] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:24 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:24.674196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.768087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5869db7-bd98-4359-ad2d-caf9785d629a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.816846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.818518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.818806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.819086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] PATH_INFO: `/v3/users/3b6a614812c04b56bea9829581803523` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.820305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21af546-9a49-4924-9350-89c86b9391fc None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:24.838485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6a614812c04b56bea9829581803523)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:24.840507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b55cc-7138-4a9e-b43c-ee86a98cbf72 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:24.842497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 109/220] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/users/3b6a614812c04b56bea9829581803523 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:24.843630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a248c-eded-4cd8-909c-9a8168e7a3d4 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.843919 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a248c-eded-4cd8-909c-9a8168e7a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.844196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a248c-eded-4cd8-909c-9a8168e7a3d4 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.845358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.845547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 110/221] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:24.847642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a248c-eded-4cd8-909c-9a8168e7a3d4 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.863337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:24.864014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.864296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.864583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.865384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:24.865695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:24.867118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60af298-7e13-4b0e-a74d-c874c8d9fb91 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:24.876209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 112/222] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:24.877554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:24.877801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:24.878130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:24.895552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:24.897376 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:24.920133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:24.972912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 113/223] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:24.973776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:24.977441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a248c-eded-4cd8-909c-9a8168e7a3d4 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.014766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.015781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.016258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.016789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.096147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e82418-ad93-4f37-bf46-ad9c9e2778b6 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.108316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.110998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.113286 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.137797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.138681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.138966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.139242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.140463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.142195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.143840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 114/224] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.144767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.189050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.238223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 111/225] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:37:25.238903 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.253040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc0969-f7e2-492a-8236-30c31238326d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.288488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.289687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.290166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.290633 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.301271 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:25.309735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:25.310622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:25.311394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:25.314066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 115/226] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:25.315064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.348743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1a484-2a37-4456-8048-b11912b7a840 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.359823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.390422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.391297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.391577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.391855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.393247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.395259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.397877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 112/227] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.403436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.441231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.443949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650e48a-b215-42b1-8950-2e85f9da37b6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.479708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.480585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.480903 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.481241 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.482705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.484321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.486707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 116/228] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.487344 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.539548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc42a37-b1cc-4d77-a369-696310b0afa1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.589857 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.590536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.590817 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.591087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.594320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:25.594320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.594320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.610776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 113/229] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.612509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.614153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851659b-5d6f-4dec-a75b-c4094f34184b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.625864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.657271 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.658325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.658607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.658884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/4a6862654bb14ea385a578ed7cf8788b/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.661674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, user_id=4a6862654bb14ea385a578ed7cf8788b, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.663547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.692887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 117/230] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:25 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/4a6862654bb14ea385a578ed7cf8788b/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:25.694918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:25.733532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:25.738846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59232255-463a-4c3d-9a2c-f2f4a237be96 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.816316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.817014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.817290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.817566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.818467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=3b6a614812c04b56bea9829581803523) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:25.818887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.820390 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.834768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 114/231] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/role_assignments?user.id=3b6a614812c04b56bea9829581803523&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.890897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff1dc6-9647-455b-986f-92ee61efc65b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:25.941029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:25.941940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:25.942230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:25.942508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:25.945940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:25.946201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:25.948781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 118/232] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:25.957740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:26.082539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff636-cbfd-43c6-93e1-3f8e9b0fd212 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:26.126009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:26.126880 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:26.127163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:26.127438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:26.128667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:26.130126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b6846-251f-48ee-88e6-d03c6905a6f6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:26.132551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 115/233] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:25 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:26.138530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:26.209714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:26.292973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfacdda-756d-4bb6-a4e6-2c01973b8a5d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:26.331581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:26.332472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:26.332787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:26.333067 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/4a6862654bb14ea385a578ed7cf8788b/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:26.335310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, user_id=4a6862654bb14ea385a578ed7cf8788b, role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:26.337153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1fae9-667e-4ffd-b7ff-9721b8a31b1a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:26.351589 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 119/234] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:26 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/4a6862654bb14ea385a578ed7cf8788b/roles/13f5475a26344e9e816475b8539030d3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:27.374151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c9f-5ca1-4502-a51b-f1e5b9d18d99 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.374671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c9f-5ca1-4502-a51b-f1e5b9d18d99 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.374781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d12c9f-5ca1-4502-a51b-f1e5b9d18d99 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.376078 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 116/235] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:27.381439 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14fba-6818-4477-a6e3-78da0203ef71 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.381850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14fba-6818-4477-a6e3-78da0203ef71 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.382019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14fba-6818-4477-a6e3-78da0203ef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.405601 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-00f14fba-6818-4477-a6e3-78da0203ef71 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:27.406805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-00f14fba-6818-4477-a6e3-78da0203ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:27.492774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 120/236] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:27.522142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8146e-57e9-415d-8a65-a6e2aba7446c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.522742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8146e-57e9-415d-8a65-a6e2aba7446c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.523253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8146e-57e9-415d-8a65-a6e2aba7446c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.525130 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 117/237] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:27.530255 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8146e-57e9-415d-8a65-a6e2aba7446c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:27.530746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.531088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.531322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.552707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:27.552707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:27.573071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:27.576324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:27.642955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 121/238] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:27.709768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:27.719341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8146e-57e9-415d-8a65-a6e2aba7446c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:27.757910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:27.759069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.759581 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.760118 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.788601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:27.802873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:27.803775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:27.804551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:27.809539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 118/239] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:27.813173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:27.881149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6509897-a3a9-4d53-b516-8636a6aea9c7 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:27.956574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:27.960279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:27.960826 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:27.961310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:27.964200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:27.966647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:27.970984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 122/240] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:27.989895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:27.990677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.013679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9ee7-a5d3-40bb-801a-88f631eb33fd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:28.051225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:28.052872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.053364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.053880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:28.054751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.055724 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:28.056242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:28.057934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:28.067803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 119/241] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:28.078693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.201678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a402fe-22d0-41f7-8d3f-43f20afa19c3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:28.210530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b5c42-1604-426e-aa82-07863c4b3fed None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:28.239215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:28.240811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.241097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.241840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] PATH_INFO: `/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.254431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:28.258413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] RBAC: Authorizing `identity:get_user(user_id=eaf56fb447ce4b81a2dd9fe0e417d98c)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:28.260073 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:28.262408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 120/242] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:28 2022] GET /identity/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:28.267445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.289069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:28.289755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.290041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.290323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.291340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:28.295168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa1352-55ce-49b5-b95c-425549ddbe94 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:28.331116 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 123/243] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:27 2022] POST /identity/v3/users => generated 293 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:37:28.421169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc684b1-668f-4dba-a174-5d49ef70f23c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:28.460775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:28.462039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.462529 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.463009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.541818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:28.544399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:28.546242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558b833-afc1-40da-b7a4-d81c0c8cf465 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:28.581320 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1c3b-aa8c-46d6-864f-e57f3e749cb5 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.581838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1c3b-aa8c-46d6-864f-e57f3e749cb5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.581928 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1c3b-aa8c-46d6-864f-e57f3e749cb5 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.583561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 124/244] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:28.599384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d22d-0f6e-4e6a-8161-cccbd8a7842b None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.599651 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d22d-0f6e-4e6a-8161-cccbd8a7842b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.599935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159d22d-0f6e-4e6a-8161-cccbd8a7842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.622461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-4159d22d-0f6e-4e6a-8161-cccbd8a7842b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:28.623758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-4159d22d-0f6e-4e6a-8161-cccbd8a7842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:28.662484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 121/245] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:28.664091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fd11f-350d-469a-91ce-847269d0ca9f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.664382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fd11f-350d-469a-91ce-847269d0ca9f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.664694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54fd11f-350d-469a-91ce-847269d0ca9f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.667753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 122/246] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:28.668589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fd11f-350d-469a-91ce-847269d0ca9f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.724272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 125/247] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:28.725326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.725614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.725888 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.742784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:28.744781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:28.748422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:28.820262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:28.824317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54fd11f-350d-469a-91ce-847269d0ca9f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:28.832511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 126/248] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:28.833792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.874488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:28.875465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:28.875756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:28.876090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:28.877330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:28.878866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:28.881299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 123/249] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:28 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:28.882182 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:28.954314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611950-bb00-40c5-b534-ad283bbf8f28 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.007258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e6d3-9590-4396-9730-f413173c7bbd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.049155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.050026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.050275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.050415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.050593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.051101 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.051391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.051672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.064627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:29.073150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:29.074087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:29.074859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 124/250] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:29.075972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.078030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.089279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:29.097278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:29.100088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 127/251] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:29.102017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.109464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.195975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199e1-9c2f-413c-9409-99ad403792a3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.241265 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.242998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.243286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.243567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.244409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=eaf56fb447ce4b81a2dd9fe0e417d98c) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:29.245065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.246989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:29.253487 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 125/252] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/role_assignments?user.id=eaf56fb447ce4b81a2dd9fe0e417d98c&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:29.254381 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.328231 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf6c58-e4ae-4d95-be41-912835e781da None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.368991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.369848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.370206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.370573 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.371448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:29.371822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.373471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf472e-b508-48c9-a07f-4aa907aadd15 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:29.383483 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 128/253] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:29.385045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615bff-9455-4914-8e63-79eccbf84ba4 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.385402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615bff-9455-4914-8e63-79eccbf84ba4 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.385749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28615bff-9455-4914-8e63-79eccbf84ba4 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.387306 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 129/254] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:29.387806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.392362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28615bff-9455-4914-8e63-79eccbf84ba4 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.399872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c910cf-f50b-4f05-9a37-ea702929942f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.451460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.452172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.452429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.452763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.454157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:29.454478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.456508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d2c49-0017-4b3b-a435-600d28870c66 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:29.463618 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.470281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 126/255] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:29.472459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.473049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.473341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.495744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:29.497645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:29.516742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28615bff-9455-4914-8e63-79eccbf84ba4 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.563076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.564277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.564578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.564867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.566726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.568540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:29.571382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 130/256] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:29.572231 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.575291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 127/257] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:29.581020 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.727251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.750134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1608e9-acfe-4a89-83d3-0634ace1ec58 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.763543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb88cc-63af-4fb6-860e-89252382dfb5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.809115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.809770 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.810058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.810331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.820836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:29.834749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.835561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.836146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.837570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] PATH_INFO: `/v3/users/3b6a614812c04b56bea9829581803523` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.839332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6a614812c04b56bea9829581803523)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.840821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:29.842613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 131/258] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/users/3b6a614812c04b56bea9829581803523 => generated 297 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:29.844382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:29.957199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e31278-9403-4cd6-9d07-361511909468 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:29.992728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:29.993647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:29.993934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:29.994208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:29.996987 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:29.998487 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.005660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:30.014236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:30.015121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:30.015905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 132/259] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:30.016547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.105096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.106569 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.137903 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 128/260] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:30.145898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.148110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48d39f-9a3e-4796-8584-7e973ee64b25 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.190799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.191692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.191975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.192318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.194955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.197929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.199655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 133/261] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:30.200421 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.328043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e56da-9068-4578-af40-0155c39dfd69 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.386511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c50f8b-2dd5-43b7-b060-bdb1f31a522a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.397248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.397930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.398218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.398496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.399252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:30.399563 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.400985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.419115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 129/262] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:30.420853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.439630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.452592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.453685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.453972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.454252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.457473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.459105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.465456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 134/263] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:30.467296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.510408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.608468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c24d-6fd4-4ec1-8615-cd8dce1cd534 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.658178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca5568-4b16-43eb-8e2c-7c383bf3a748 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.666681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.667640 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.667934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.668212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/3b6a614812c04b56bea9829581803523/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.672862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, user_id=3b6a614812c04b56bea9829581803523, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.674341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.694233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 130/264] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:30 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/3b6a614812c04b56bea9829581803523/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:30.697441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.738204 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.739146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.739437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.739772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] PATH_INFO: `/v3/users/4a6862654bb14ea385a578ed7cf8788b` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.744513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a6862654bb14ea385a578ed7cf8788b)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.746436 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:30.749174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 135/265] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/users/4a6862654bb14ea385a578ed7cf8788b => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:30.755580 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:30.892943 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.907277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e6266-1b6b-45d5-b19b-6e91ead3f918 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.919928 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:30.951244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928405-ebf1-48ec-bf29-0d9092ce24f0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:30.984302 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:30.985042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:30.985334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:30.985612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:30.986475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=4a6862654bb14ea385a578ed7cf8788b) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:30.986793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:30.988308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:31.001101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 131/266] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/role_assignments?user.id=4a6862654bb14ea385a578ed7cf8788b&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:31.012037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:31.012764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.013057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.013341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.108479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:31.110056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:31.179768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:31.240202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 136/267] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:31.246738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:31.430072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd6d75-d560-4a0e-b778-3901544872fb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:31.479920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:31.480832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.481114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.481388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.482614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:31.484091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c6b20-9a14-4da4-b7eb-9d5115dbcb5b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:31.486300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 132/268] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:31 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:31.492628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:31.514536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed820-668b-49a1-aeb9-42026fc88c70 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.515430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed820-668b-49a1-aeb9-42026fc88c70 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.515881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eed820-668b-49a1-aeb9-42026fc88c70 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.517833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 133/269] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:31.523764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.524298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.524778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.555258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:31.556881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:31.617644 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:31.672246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 134/270] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:31.674414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a753c7-9f5d-45da-bb2c-c2ca3b027fce None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:31.725557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:31.746989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:31.747666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.747953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.748235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.750546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=4a6862654bb14ea385a578ed7cf8788b) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:31.750871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:31.753126 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf31c-c9f8-4bc2-b98e-8149b9f4d9b8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:31.767958 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 137/271] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:31 2022] GET /identity/v3/role_assignments?user.id=4a6862654bb14ea385a578ed7cf8788b&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=13f5475a26344e9e816475b8539030d3 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:31.769654 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed049f5-fbb4-4df0-a077-952207588bc5 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.770129 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed049f5-fbb4-4df0-a077-952207588bc5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.771277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed049f5-fbb4-4df0-a077-952207588bc5 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.773279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 138/272] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:31.779898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.781180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.781551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.827003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:31.828237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:31.878305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:31.924763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0fd97-63a2-47ec-a7b6-045c97c76886 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:31.963966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:31.965069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:31.965531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:31.965956 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:31.967444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:31.969178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:31.971473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 135/273] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:31.978447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:31.982160 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 139/274] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:32.024476 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.035284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.140512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae390b7-3873-4033-8ca2-7690a5ee0731 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.164363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba03c651-0b48-4b83-b9cb-7aef35beec7d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.186747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.187843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.188997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.189505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.190756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:32.194238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62d474-1d40-416c-aceb-30919ab16ce7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:32.205493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.206309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.206593 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.206868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.219567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:32.222682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 136/275] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:37:31 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:32.224324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403d6dd-e392-454a-8e78-b0034de4efe3 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.224786 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403d6dd-e392-454a-8e78-b0034de4efe3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.225268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403d6dd-e392-454a-8e78-b0034de4efe3 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.226758 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 137/276] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:32.227638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:32.228525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:32.229461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:32.236607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 140/277] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:32.238446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.238734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.239726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.241728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.258629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:32.259861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:32.268914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.350774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 138/278] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:32.379092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.390508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83001ac-78af-477c-815c-a9159f8d8f65 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.413019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.444807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.445752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.446191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.446468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.447221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:32.447530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:32.449422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:32.459316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 141/279] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:32.464955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.515570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59b96d-0bae-48fc-814d-d929f1524bc3 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.568100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.568989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.569279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.569558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.579200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.584025 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:32.595450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:32.596452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:32.598082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 139/280] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:32.603717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.620435 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff0832-57e6-4a9d-82b6-fc578f3ca10f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.669493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.672165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.672661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.673119 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] PATH_INFO: `/v3/users/cec1a349eed046ceac02db1c36dfc06c` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.691048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cec1a349eed046ceac02db1c36dfc06c)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:32.692986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:32.695551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 142/281] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/users/cec1a349eed046ceac02db1c36dfc06c => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:32.701636 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.710395 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.735746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af8a3-f15b-4f67-89f7-8b7e3107eca7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.779237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.779925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.780808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.781188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.782285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:32.782690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:32.784359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:32.797042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 140/282] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:32.809655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:32.828615 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c643325-d852-4135-83e6-3ee62a01e6d9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:32.903442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:32.904113 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:32.904408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:32.904718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:32.905555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, user.id=cec1a349eed046ceac02db1c36dfc06c, scope.system=all) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:32.905875 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:32.909882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:32.916870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:32.923666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 143/283] 10.222.0.21 () {56 vars in 1308 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/role_assignments?user.id=cec1a349eed046ceac02db1c36dfc06c&scope.system=all&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:32.978310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d84a7-b516-4508-b5a9-af03bbb61dd3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:33.017610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:33.018511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:33.018794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:33.019068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] PATH_INFO: `/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:33.020588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] RBAC: Authorizing `identity:get_user(user_id=eaf56fb447ce4b81a2dd9fe0e417d98c)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:33.022433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:33.024540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 141/284] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:32 2022] GET /identity/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:33.030311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:33.142360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae990c-fc13-4607-8ff6-5fdb6075b3ea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:33.178560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:33.179440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:33.179727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:33.180006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:33.181262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:33.182717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06f97d-143f-45fe-b21c-cf1873f7ef0e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:33.184740 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 144/285] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:33 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:33.198432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:33.215136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:33.307896 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c39556-b663-4e48-aa4d-6939b1da86cb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:33.344591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:33.345492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:33.345779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:33.346060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/eaf56fb447ce4b81a2dd9fe0e417d98c/roles/865913f1b8fb4d0db423c6c32d8f08fa` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:33.348061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, user_id=eaf56fb447ce4b81a2dd9fe0e417d98c, role_id=865913f1b8fb4d0db423c6c32d8f08fa)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:33.349842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5383e55-c6cf-44a1-8592-bb94569ca2a1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:33.365267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 142/286] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:37:33 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/users/eaf56fb447ce4b81a2dd9fe0e417d98c/roles/865913f1b8fb4d0db423c6c32d8f08fa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:33.834865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04f54a-9200-4da4-b563-dc369985cde8 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:33.835220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04f54a-9200-4da4-b563-dc369985cde8 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:33.835437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04f54a-9200-4da4-b563-dc369985cde8 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:33.837197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 145/287] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:33.843256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:33.844216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:33.844758 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:33.873698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:33.875269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:33.970899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 143/288] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:34.003459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04f54a-9200-4da4-b563-dc369985cde8 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:34.091555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:34.131815 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04f54a-9200-4da4-b563-dc369985cde8 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:34.168341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:34.169192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:34.169848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:34.170135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:34.181637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:34.189465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:34.190338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:34.191139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 146/289] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:34.197514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:34.200888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:34.309056 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e0426-87e1-4a53-83bf-a6796ec4f1c8 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:34.343990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:34.344696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:34.344987 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:34.345271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:34.346023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:34.346332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:34.347719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:34.357253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 144/290] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:34.362826 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:34.468386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46de83b-dbba-4fa5-bc69-da55aff72cfd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:34.496461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:34.506025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:34.506963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:34.507258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:34.507544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] PATH_INFO: `/v3/users/3b6a614812c04b56bea9829581803523` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:34.508795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] RBAC: Authorizing `identity:get_user(user_id=3b6a614812c04b56bea9829581803523)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:34.510295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:34.512105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 147/291] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:34 2022] GET /identity/v3/users/3b6a614812c04b56bea9829581803523 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:34.517486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:34.587851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:34.625150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ac7f7c-5f8e-4c5c-8717-84a0a5c500a8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:34.662291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:34.663501 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:34.664150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:34.664594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:34.736462 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:34.738274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:34.803558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:34.846975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 145/292] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:34.853778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:34.963209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec666-25a3-4c49-80c7-30a907f20f99 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:34.998605 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:34.999750 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.000180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.000628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.001956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:35.003561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95b19f-5c6d-4756-a645-b0fad045e39d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:35.006401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 148/293] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:34 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:35.013698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:35.146120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9385d59-b542-4057-b41e-e1f0f95a5264 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:35.184458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:35.185424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.185855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.186272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.187256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=3b6a614812c04b56bea9829581803523) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:35.187728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:35.189360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6e564-2022-4d3a-982d-7faa138c2bd6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:35.197411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 146/294] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:35 2022] GET /identity/v3/role_assignments?user.id=3b6a614812c04b56bea9829581803523&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:35.413085 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a9f9-ebbb-4e85-90f3-0975d11bb138 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.413928 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a9f9-ebbb-4e85-90f3-0975d11bb138 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.414353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7a9f9-ebbb-4e85-90f3-0975d11bb138 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.416218 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 149/295] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:35.421684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22340fcd-e41e-43d4-88d2-f123d5211cb9 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.422203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22340fcd-e41e-43d4-88d2-f123d5211cb9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.422678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22340fcd-e41e-43d4-88d2-f123d5211cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.441315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-22340fcd-e41e-43d4-88d2-f123d5211cb9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:35.442696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-22340fcd-e41e-43d4-88d2-f123d5211cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:35.471512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:35.526428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 147/296] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:35.551650 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7a9f9-ebbb-4e85-90f3-0975d11bb138 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:35.623466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d218e-ed8e-4120-8efd-00d7015c02b9 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.624151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d218e-ed8e-4120-8efd-00d7015c02b9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.624486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d218e-ed8e-4120-8efd-00d7015c02b9 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.626418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 148/297] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:35.633425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ac16-fa4f-49f9-9993-b8ba232e3283 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.633425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ac16-fa4f-49f9-9993-b8ba232e3283 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.633425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91ac16-fa4f-49f9-9993-b8ba232e3283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.636896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:35.665390 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9c91ac16-fa4f-49f9-9993-b8ba232e3283 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:35.667224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9c91ac16-fa4f-49f9-9993-b8ba232e3283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:35.689191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7a9f9-ebbb-4e85-90f3-0975d11bb138 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:35.737984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:35.739373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.739658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.740675 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.752348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f30f6fd1-60ac-471f-b886-d5d3cf76357a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:35.761858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:35.762755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 150/298] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:35.768912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fe7f-4988-4bb2-93d3-4e529bec1aa6 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.769715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fe7f-4988-4bb2-93d3-4e529bec1aa6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.769715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241fe7f-4988-4bb2-93d3-4e529bec1aa6 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.769715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 151/300] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:35.769715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241fe7f-4988-4bb2-93d3-4e529bec1aa6 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:35.772643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 149/300] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:35.780133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.780420 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.780782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.798868 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:35.800047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:35.875066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 150/301] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:35.876755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:35.890857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241fe7f-4988-4bb2-93d3-4e529bec1aa6 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:35.905886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:35.917791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:35.930228 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:35.930899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:35.931609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:35.931894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:35.932698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:35.933016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:35.935066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:35.949168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 152/302] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:35.949168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.009412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf824ad9-edb0-4949-bd3d-a98bef0ab1d3 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.049840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.050762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.051689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.052096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.053393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.055258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.056831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 151/303] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:36.057604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.062545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe140dc0-06a2-49c9-98b7-d00bc0a8a685 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.106522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.107520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.107813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.108135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.120471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:36.128359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:36.129851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:36.130621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:36.131226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 153/304] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:36.132077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.161592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.180036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.180524 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef6463-9698-4282-ae06-d03f0de0f3de None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.220155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.221177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.221409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.221667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] PATH_INFO: `/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.235775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d8ff8cfe4443758ce0ab33e5e51279)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.237315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.239558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 152/305] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:36.242791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9add7d-bd57-488c-960b-459eb71f3471 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.244878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.279180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.279849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.280148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.280426 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] PATH_INFO: `/v3/groups` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.281158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.282601 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.297563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 154/306] 10.222.0.21 () {60 vars in 1134 bytes} [Wed May 25 13:37:36 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:37:36.305641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.369538 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12259f98-3170-4282-b934-a7184ae1c8f3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.415247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.415940 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.416226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.416502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.417306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:36.417622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.419043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1bff4-8b45-4ab2-9000-096fe1896c0c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.436282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 153/307] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:36.436282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545ca8-ce61-4a65-9268-235bb81bef65 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.436282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545ca8-ce61-4a65-9268-235bb81bef65 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.436282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53545ca8-ce61-4a65-9268-235bb81bef65 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.436282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 154/308] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:36.439036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.439335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.439618 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.451977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.452596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ae26f-928c-4e38-a9a4-e87afc98882a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.460681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.461127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:36.462175 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:36.497581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.498252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.498543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.498822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.499667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, user.id=c8d8ff8cfe4443758ce0ab33e5e51279, scope.system=all) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:36.499983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.502272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.509658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 155/309] 10.222.0.21 () {56 vars in 1308 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/role_assignments?user.id=c8d8ff8cfe4443758ce0ab33e5e51279&scope.system=all&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:36.509944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.547621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 155/310] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:36.572991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.645884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184decd-a50a-442a-a6e8-73999064b746 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.687237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.688050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.688334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.688663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] PATH_INFO: `/v3/users/cec1a349eed046ceac02db1c36dfc06c` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.690035 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] RBAC: Authorizing `identity:get_user(user_id=cec1a349eed046ceac02db1c36dfc06c)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.691572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.694329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 156/311] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/users/cec1a349eed046ceac02db1c36dfc06c => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:36.700918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.731039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f1b66-1d6b-42ce-8d1c-d8dcb0071274 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.758283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.776152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.776991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.777277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.777554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.780682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:36.795489 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:36.807143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:36.808052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:36.808860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:36.810091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 156/312] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:36.815573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:36.831697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24efc69-3791-4519-a397-ad71990badd7 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.877898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28756bce-07af-4454-b495-feba2374070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.879893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756bce-07af-4454-b495-feba2374070e None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.880186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756bce-07af-4454-b495-feba2374070e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.880932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28756bce-07af-4454-b495-feba2374070e None admin] PATH_INFO: `/v3/system/users/cec1a349eed046ceac02db1c36dfc06c/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.882800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28756bce-07af-4454-b495-feba2374070e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cec1a349eed046ceac02db1c36dfc06c, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.885030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28756bce-07af-4454-b495-feba2374070e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:36.897113 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 157/313] 10.222.0.21 () {58 vars in 1334 bytes} [Wed May 25 13:37:36 2022] PUT /identity/v3/system/users/cec1a349eed046ceac02db1c36dfc06c/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:36.951323 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251d6f5-7cb4-433c-98e3-0708f5bde259 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:36.992538 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:36.993259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:36.993548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:36.993832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:36.994704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:36.995026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:36.996600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:37.007298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 157/314] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:37.012703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28756bce-07af-4454-b495-feba2374070e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:37.128350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28756bce-07af-4454-b495-feba2374070e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:37.131083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:37.168762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:37.169685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:37.169977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:37.170263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] PATH_INFO: `/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:37.171694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] RBAC: Authorizing `identity:get_user(user_id=eaf56fb447ce4b81a2dd9fe0e417d98c)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:37.173225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:37.175294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 158/315] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:37 2022] GET /identity/v3/users/eaf56fb447ce4b81a2dd9fe0e417d98c => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:37.180685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:37.243573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:37.288633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22c22c-49a6-4db3-ba9d-2afa9cc9ce55 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:37.323446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:37.324109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:37.324396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:37.324709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:37.394176 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:37.396894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:37.454731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:37.502643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 158/316] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:37.509472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:37.615437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef094-4f28-4222-99e7-fafaa12f92af None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:37.653270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:37.654137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:37.654425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:37.654693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:37.655881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:37.657354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f356db-8219-4996-a68f-f3cf1d36ffa4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:37.659448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 159/317] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:37 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:37.665425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:37.771924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695b1e1-254b-4f04-b480-35d509809b18 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:37.809610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:37.810609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:37.811134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:37.811451 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:37.812293 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, scope.project.id=8dc4238400014620b44e47d1b740b7b7, user.id=eaf56fb447ce4b81a2dd9fe0e417d98c) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:37.812634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:37.814087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3261a2c-23fb-4a72-ab06-9abac95ced2e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:37.821188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 159/318] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:37:37 2022] GET /identity/v3/role_assignments?user.id=eaf56fb447ce4b81a2dd9fe0e417d98c&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:38.742787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424e9b0-55c6-4503-8357-6e17a7c719f6 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:38.743546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424e9b0-55c6-4503-8357-6e17a7c719f6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:38.743546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424e9b0-55c6-4503-8357-6e17a7c719f6 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:38.744993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 160/319] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:38.751138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564d8d-9a3f-41ad-9bd7-6ea7569666cb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:38.751651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564d8d-9a3f-41ad-9bd7-6ea7569666cb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:38.752206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61564d8d-9a3f-41ad-9bd7-6ea7569666cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:38.780442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-61564d8d-9a3f-41ad-9bd7-6ea7569666cb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:38.781745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-61564d8d-9a3f-41ad-9bd7-6ea7569666cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:38.802417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:38.862131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 160/320] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:38.877094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b8-b793-4c6b-a9a6-375bf8629066 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:38.877567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b8-b793-4c6b-a9a6-375bf8629066 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:38.877982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b8-b793-4c6b-a9a6-375bf8629066 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:38.880471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 161/321] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:38.885383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e5b8-b793-4c6b-a9a6-375bf8629066 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:38.887516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:38.887930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:38.888330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:38.907617 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:38.908842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:38.934586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.007115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 161/322] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:39.024428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e5b8-b793-4c6b-a9a6-375bf8629066 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.030094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.084855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.087976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.088695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.089387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.092491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.096702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.100992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 162/323] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:39.107944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.142133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.163181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cf3c6-3898-4fff-afea-56213da86c4f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.218949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.220290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.220840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.221332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.239879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:39.252821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66aabae-68bd-4185-acb7-2424e94b68df None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:39.255843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:39.256815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:39.257806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:39.261421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 162/324] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:39.263765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.268095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.306331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.307022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.307314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.307592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] PATH_INFO: `/v3/groups` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.308294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.309799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bf760-10ea-4122-b083-5533c92aba53 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.335933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 163/325] 10.222.0.21 () {60 vars in 1134 bytes} [Wed May 25 13:37:39 2022] POST /identity/v3/groups => generated 226 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:37:39.335933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20e919-c8ac-44c1-8686-87c204869a19 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.335933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20e919-c8ac-44c1-8686-87c204869a19 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.335933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20e919-c8ac-44c1-8686-87c204869a19 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.335933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 164/326] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:39.339409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.339713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.339999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.359409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:39.361024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:39.380315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bedda-1433-4603-96cc-6901d16c9b8d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.419971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.420754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.421043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.421326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.422112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:39.422431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.423857 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.433230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 163/327] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:39.439316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.444729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.447206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 165/328] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:39.470988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.557592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da919a-0792-49ed-b67e-4b4982f72f94 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.598459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.603797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f459ab-e842-4ced-b65d-814e77a30a24 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.611340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.612229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.612515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.612822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] PATH_INFO: `/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.614869 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d8ff8cfe4443758ce0ab33e5e51279)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.616349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.618741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 164/329] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:39.624969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.656155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.657361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.657789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.658078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.673163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:39.682614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:39.683543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:39.684546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 166/330] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:39.689413 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:39.714460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:39.759058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cea636-b1a8-4487-a8bb-46fb8d481bb9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.804147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.805059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.805339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.805616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] PATH_INFO: `/v3/system/users/c8d8ff8cfe4443758ce0ab33e5e51279/roles/865913f1b8fb4d0db423c6c32d8f08fa` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.808013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c8d8ff8cfe4443758ce0ab33e5e51279, role_id=865913f1b8fb4d0db423c6c32d8f08fa)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.809449 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.821058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 165/331] 10.222.0.21 () {58 vars in 1334 bytes} [Wed May 25 13:37:39 2022] PUT /identity/v3/system/users/c8d8ff8cfe4443758ce0ab33e5e51279/roles/865913f1b8fb4d0db423c6c32d8f08fa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:39.843175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9e78f-b1c5-41fa-9dc3-0f5d9b1d00dd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:39.887384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:39.888399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:39.888891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:39.889351 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:39.890378 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:39.890849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:39.892543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:39.904045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 167/332] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:39.909794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:40.032193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ae7a5-2dee-4f01-9d12-7aad2183dab6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:40.069891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:40.071014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:40.071468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:40.071893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] PATH_INFO: `/v3/users/cec1a349eed046ceac02db1c36dfc06c` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:40.073285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] RBAC: Authorizing `identity:get_user(user_id=cec1a349eed046ceac02db1c36dfc06c)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:40.074978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d51afc-1491-4d10-afda-d1896f6aec60 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:40.077098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 166/333] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:39 2022] GET /identity/v3/users/cec1a349eed046ceac02db1c36dfc06c => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:40.082805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:40.104536 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:40.199789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a5287-5a3e-4256-a558-0f11adfdf754 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:40.242696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:40.243371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:40.243658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:40.243936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:40.244857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] RBAC: Adding query filter params (role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, user.id=cec1a349eed046ceac02db1c36dfc06c, scope.system=all) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:40.245176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:40.246669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053eca-0351-429f-838c-86f69af62921 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:40.261556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 168/334] 10.222.0.21 () {56 vars in 1308 bytes} [Wed May 25 13:37:40 2022] GET /identity/v3/role_assignments?user.id=cec1a349eed046ceac02db1c36dfc06c&scope.system=all&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:42.476127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125dd5-8a96-465c-a1d9-58418d3cabb6 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.477238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125dd5-8a96-465c-a1d9-58418d3cabb6 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.477747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125dd5-8a96-465c-a1d9-58418d3cabb6 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.479978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 167/335] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:42.488512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.489671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.490160 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.509915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:42.511633 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:42.519304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f7e8e-593b-4433-ac5a-b888bf3a5b22 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.519638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f7e8e-593b-4433-ac5a-b888bf3a5b22 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.519895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f7e8e-593b-4433-ac5a-b888bf3a5b22 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.525377 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 168/336] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:42.529547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.529846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.530130 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.540922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:42.560052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:42.561259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:42.649602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 169/337] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:42.695225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:42.696075 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 169/338] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:42.707560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:42.724414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:42.851491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c56c0f-b917-4ad3-b029-b06faacf74d8 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:42.867190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4951d71-3969-4481-bc83-5712b94cdadf None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:42.919017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:42.919832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.920119 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.920398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.929328 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:42.930144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.930425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.930698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.936899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:42.941331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-cdd43ccc-da96-4bee-9e3d-be54792aa043 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:42.949215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:42.950156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:42.950971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 170/339] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:42.955575 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c925-389e-4991-8ec4-f710281143d1 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.960223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:42.965032 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c925-389e-4991-8ec4-f710281143d1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.965335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953c925-389e-4991-8ec4-f710281143d1 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.967161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 171/340] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:42 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:42.967813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c925-389e-4991-8ec4-f710281143d1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-ffb951e1-530c-41be-8c56-10a863acb2a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:42.972726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:42.974813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:42.976800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 170/341] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:42.978415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:42.978415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:42.978415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:42.996442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:42.997739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:43.080184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 171/342] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:43.081068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.083834 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.142613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953c925-389e-4991-8ec4-f710281143d1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.199630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.200312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.200637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.201026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.201893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:43.202210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.203608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.217550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc729829-5dcd-44c1-aaa7-a074afa0e12f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.218920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 172/343] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:43.219768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.246078 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.259116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.259965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.260245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.260610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.261447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:43.261758 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.263167 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.274442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 172/344] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:43.275299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.429218 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0fb16-2dbc-4956-b3a0-5221e9242217 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.459882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d843ddb-5db1-4630-b048-9353ae299c91 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.487063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.506872 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.507697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.507984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.508259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.526943 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:43.531367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.532341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.532714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.533059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] PATH_INFO: `/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.534332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d8ff8cfe4443758ce0ab33e5e51279)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.536153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6c653b5f-093b-46cb-935e-a6223879291b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:43.537583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:43.538473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:43.539270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:43.539952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 173/345] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/users/c8d8ff8cfe4443758ce0ab33e5e51279 => generated 293 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:43.539952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.542254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 173/346] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:43.543842 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03b16-8bab-48ed-a837-e19b434b6b31 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.544339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03b16-8bab-48ed-a837-e19b434b6b31 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.544912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f03b16-8bab-48ed-a837-e19b434b6b31 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.546737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 174/347] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:43.547657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f03b16-8bab-48ed-a837-e19b434b6b31 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.627291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.680734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d216d2-2a39-40be-9853-22c2085050f9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.685325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f03b16-8bab-48ed-a837-e19b434b6b31 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.720983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.721958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.723090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.723556 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.727192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.727854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.728147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.728425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.729301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] RBAC: Adding query filter params (role.id=865913f1b8fb4d0db423c6c32d8f08fa, user.id=c8d8ff8cfe4443758ce0ab33e5e51279, scope.system=all) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:43.729608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.731063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.752377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 175/348] 10.222.0.21 () {56 vars in 1308 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/role_assignments?user.id=c8d8ff8cfe4443758ce0ab33e5e51279&scope.system=all&role.id=865913f1b8fb4d0db423c6c32d8f08fa => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:43.753742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:43.821899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.823501 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5f2ef-fcb9-4caf-820e-45a653002c18 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.826407 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.904499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:43.906340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ec9f4-e3b9-4e04-82c3-c8fa120e6c69 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:43.953459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:43.954401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.954859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.955288 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.956269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:43.957268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:43.958873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fc2cc-e12a-4f16-a009-04c238e1ec91 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:43.968759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 174/349] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:43.969627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 176/350] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:43.970483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.970919 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.971243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd398-5798-4291-a8d8-52dc59acd190 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:43.971513 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.971718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd398-5798-4291-a8d8-52dc59acd190 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:43.972145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcd398-5798-4291-a8d8-52dc59acd190 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:43.973881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 177/351] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:43.974828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd398-5798-4291-a8d8-52dc59acd190 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.000028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:44.001297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:44.079333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 175/352] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:44.080195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.085553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcd398-5798-4291-a8d8-52dc59acd190 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.095200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.124942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.125847 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.126158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.126405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.127632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:44.129870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c59be-b5c4-48d2-8b88-b5a71f6f079b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:44.136597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 178/353] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:43 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:44.137990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.138280 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.138554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.164804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:44.166407 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:44.198876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.224657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96af19d-c5fc-4c07-a163-92c42a2c530d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.273650 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.274561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.274678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 179/354] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:44.274759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.274931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.280657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.386219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:44.388778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:44.409137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e51010-6c59-4663-a540-7f8a41184bd1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.419694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.462746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.464002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.464290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.464591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.480979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:44.489767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:44.490715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 180/355] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:44.491910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.509098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.533027 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 176/356] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:44.535902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.626136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336649bc-a88f-42ab-abaf-97bc5c56fb55 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.655223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4fc44-5fef-48ed-992a-b3c854b58659 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.668187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.669158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.669452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.669736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.677537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:44.679059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:44.680832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 181/357] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:44.681613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.700714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.701526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.701807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.702234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.716087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:44.718641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:44.725885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:44.726844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:44.727628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:44.729997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 177/358] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:44.731164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.789618 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:44.806545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feac0cc-acf0-4c9a-bd91-48da7419398f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.853369 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.854059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.854349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.854628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.855386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:44.855697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:44.857171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab2d2a-14a9-411f-9631-23a1c2cfa58d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:44.858820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcaa0-0144-4904-9fd1-540d94de66d2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:44.867290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 182/359] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:44.868721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa150-f22d-44a4-a250-7c67e71a10f6 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.869018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa150-f22d-44a4-a250-7c67e71a10f6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.869291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365aa150-f22d-44a4-a250-7c67e71a10f6 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.874632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 183/360] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:44.874632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aa150-f22d-44a4-a250-7c67e71a10f6 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.900416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:44.901364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:44.901654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:44.901932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:44.903174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:44.904845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:44.906772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 178/361] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:44.907398 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:44.993432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.010489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365aa150-f22d-44a4-a250-7c67e71a10f6 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.028640 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e55ab-919b-4a27-b094-db57d1c5915e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.044602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.060194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.060875 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.061345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.061768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.062749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=7ab3476e508d41e482b975ae625333e3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:45.063218 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.064678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.081758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 184/362] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:45.081758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.084666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.087924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.087924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.087924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.087924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:45.087924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.088548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d998e9-512d-4a72-a050-7ff12eeb9bb7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.098277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 179/363] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:45.100062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.100360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.100700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.120980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:45.122508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:45.217718 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 180/364] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:45.218935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.228331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac49152-1ce8-4039-ae18-52ddb34e7952 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.276020 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.276652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.276945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.277224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.309781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.348346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2bc0e-357a-4af9-9846-f60a110ff089 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.350762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.378246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.379935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.402063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.403209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.404295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.404295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.405183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.406655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.408465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 181/365] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:45.410603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.506277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 185/366] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:45.508839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.537335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b91989-7d69-4805-9208-c27773e6ccc7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.579266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.579628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.580479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.581104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.581820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.645384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.661423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df30680-36a7-487d-854b-2d8f92083ca7 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.686914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.688381 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.719589 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.720400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.720751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.721037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.738707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:45.745818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:45.746717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:45.747500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 186/367] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:45.748150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.834155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 182/368] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:45.834947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.889423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b47dd7-ebec-4f23-8c3f-5cb504b903d1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:45.915648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.924521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:45.951763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:45.954818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:45.955109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:45.955393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:45.957120 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=7ab3476e508d41e482b975ae625333e3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:45.957454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:45.969840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:45.969840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 187/369] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=13f5475a26344e9e816475b8539030d3 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:45.969840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:45.984185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473b239-8654-4a95-97aa-6f59b125f6ab None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.027876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.028791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.029776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.030068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.031773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.033760 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.038164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 183/370] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.039333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.110242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf7dc-42ae-4188-ba58-79e53c9b4b8c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.159160 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.160296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.160835 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.161332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.163391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:46.163999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.165700 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.175345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 188/371] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.176657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.185425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c853f-2dce-4b9c-83e4-ed489270d749 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.230366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.231656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.232410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.233020 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.233599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.235152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.237513 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.240752 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 184/372] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.241868 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.254198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.294693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124563-206c-454b-90ab-f281cc1b5606 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.333119 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.335646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.335929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.336213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.340075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.340075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.348892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 189/373] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.349717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.371721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af39ee-ec72-478e-89f2-c553404ba4dd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.410029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.410710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.410997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.411275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.487881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.489596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.490276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fd331-cdb1-4292-850f-fa36ee8c03e7 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.500290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.506184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.545894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.546760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.547043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.547315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.549202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.551524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.557657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 190/374] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.557657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.634799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 185/375] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:46.636912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.691895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405d8d-bc83-4d86-b134-5ab08da88ab3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.733794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.734748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.735211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.735691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.736956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=8dc4238400014620b44e47d1b740b7b7) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:46.738217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.739883 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.746313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 191/376] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=13f5475a26344e9e816475b8539030d3 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.747568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:46.755425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1d61d-6dce-43c9-9981-085383a7f7cb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.769680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.798449 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.799055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.799345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.799686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.801276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=8dc4238400014620b44e47d1b740b7b7) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:46.801602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.803225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.814778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 186/377] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.833655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:46.929117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557260be-79e3-4ee2-bc49-8e869ec66d52 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:46.980641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:46.981598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:46.981891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:46.982174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] PATH_INFO: `/v3/projects/d26306eca6ff451eb60119f9e4cf5383` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:46.983522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] RBAC: Authorizing `identity:get_project(project_id=d26306eca6ff451eb60119f9e4cf5383)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:46.985172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:46.987245 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 192/378] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/projects/d26306eca6ff451eb60119f9e4cf5383 => generated 344 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:46.993197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:47.113562 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6e896-6bb4-49be-ba6e-8c1e4d57dce3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:47.152012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:47.152925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:47.153281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:47.153567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] PATH_INFO: `/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:47.160376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] RBAC: Authorizing `identity:get_group(group_id=5abb7ae1a7e04bbfbc618a909fdeeb1d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:47.161958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e78e7-e616-45a4-846e-81a6b02835a6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:47.164046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 187/379] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:46 2022] GET /identity/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:47.173777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:47.321431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9335d14-819e-4204-a3cf-c101897cc19f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:47.340555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:47.383169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:47.383844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:47.384135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:47.384416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:47.385298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] RBAC: Adding query filter params (group.id=5abb7ae1a7e04bbfbc618a909fdeeb1d, role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=d26306eca6ff451eb60119f9e4cf5383) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:47.385620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:47.387067 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8b5d3-d0f8-457b-b981-2d9f0d0acdcb None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:47.398049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 193/380] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:47 2022] GET /identity/v3/role_assignments?group.id=5abb7ae1a7e04bbfbc618a909fdeeb1d&scope.project.id=d26306eca6ff451eb60119f9e4cf5383&role.id=698416d22e5b4c278ff780c5eef36ede => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:47.591935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754d0ac-0ec9-4766-a815-b1b547606e51 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:47.592285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754d0ac-0ec9-4766-a815-b1b547606e51 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:47.592465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754d0ac-0ec9-4766-a815-b1b547606e51 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:47.593895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 188/381] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:47.603159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:47.603784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:47.604262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:47.626860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:47.628691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:47.699682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 194/382] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:47.746321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754d0ac-0ec9-4766-a815-b1b547606e51 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:47.801728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:47.892642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754d0ac-0ec9-4766-a815-b1b547606e51 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:47.954782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:47.955621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:47.956116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:47.956411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:47.973697 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:47.981787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:47.982746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:47.983539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 189/383] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:47.991363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:48.106986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46aa634-c0a0-4a06-a680-db4754a13f39 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:48.110272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:48.146859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:48.147453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.147735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.148010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.148783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:48.149092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:48.150471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:48.159684 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 195/384] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:48.165151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:48.282519 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:48.289438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22025134-074d-4b8f-ba10-deb92d06d9c6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:48.328211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:48.329125 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.329408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.329681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.331244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:48.332889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:48.335178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 190/385] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:48 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:48.341859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:48.458634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64229196-7858-49b6-acbb-bd18a8e60898 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:48.496109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:48.497084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.497370 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.497643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.498934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:48.500433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f07c41-5656-4773-b011-7f055ae3d6e5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:48.503183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 196/386] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:48 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:48.509780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:48.627425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985b0b-1645-4047-ad7c-c9761fa991c2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:48.672000 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:48.673294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.673926 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.674930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/groups/d9010b3c5d294d7e9574e812de5442f0/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.681864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, group_id=d9010b3c5d294d7e9574e812de5442f0, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:48.683553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e321af-c8c7-4f82-a85b-97cbd8a957a5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:48.698223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 191/387] 10.222.0.21 () {58 vars in 1442 bytes} [Wed May 25 13:37:48 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/groups/d9010b3c5d294d7e9574e812de5442f0/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:48.813163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1d223-fe69-4d5b-b510-9f6adaf81d14 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.813669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1d223-fe69-4d5b-b510-9f6adaf81d14 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.813669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1d223-fe69-4d5b-b510-9f6adaf81d14 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.816089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 197/388] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:48.822067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:48.823060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:48.823545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:48.853774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:48.855349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:48.881484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:48.950203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 192/389] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:48.974074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1d223-fe69-4d5b-b510-9f6adaf81d14 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.021759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:49.101369 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1d223-fe69-4d5b-b510-9f6adaf81d14 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.146091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:49.146922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.147209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.147488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.160679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:49.173010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:49.173906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:49.176256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 198/390] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:49.180951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.320385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05db6b-f0cb-4428-8b2d-90583c8c6a69 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.359943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:49.360612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.360914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.361299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.362823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:49.363145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:49.364765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:49.374010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 193/391] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:49.380238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.492977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:49.498821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af35a11-2391-449b-a27f-4e98aa7e9ebd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.536852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:49.538013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.538449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.538876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.540544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:49.543557 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980d128-b783-49c3-be4e-fe896f13f743 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:49.547144 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 199/392] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:49 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:49.554996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.582740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:49.662350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8d888-ba25-4ac5-85ab-0ec1442902ce None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.663014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8d888-ba25-4ac5-85ab-0ec1442902ce None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.663525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8d888-ba25-4ac5-85ab-0ec1442902ce None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.665452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 200/393] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:49.670951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.671570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.672070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.690513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:49.692075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:49.707474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc214fd0-8fe7-450f-b991-22e6c8b5cfce None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.767118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 201/394] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:49.790625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.798664 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:49.799583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.799886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.800466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.801535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:49.803044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:49.804901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 194/395] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:49 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:49.810206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:49.915514 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:49.932686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960b62c-b329-4391-bb26-108cc19b4da3 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.947565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:49.971856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49059ec6-43d4-4957-ba79-578628c019da None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:49.980504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:49.981348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:49.981629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:49.981947 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:49.999227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-88b9f444-2e45-41b7-a07c-79099f659b54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:50.010298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:50.011283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:50.012157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:50.016694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 202/396] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:50.019353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553183-b371-44d6-bea7-76125ec69d88 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.019663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553183-b371-44d6-bea7-76125ec69d88 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.019929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553183-b371-44d6-bea7-76125ec69d88 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.024956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 203/397] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:50.026997 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553183-b371-44d6-bea7-76125ec69d88 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.034297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.035246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.035539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.035828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3/groups/d9010b3c5d294d7e9574e812de5442f0/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.038359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ab3476e508d41e482b975ae625333e3, group_id=d9010b3c5d294d7e9574e812de5442f0, role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.039843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cfd20-0faa-4b25-84eb-02b77a5c9bc8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.057466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 195/398] 10.222.0.21 () {58 vars in 1442 bytes} [Wed May 25 13:37:49 2022] PUT /identity/v3/projects/7ab3476e508d41e482b975ae625333e3/groups/d9010b3c5d294d7e9574e812de5442f0/roles/13f5475a26344e9e816475b8539030d3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:37:50.059012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.059308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.059652 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.078955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:50.081124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:50.150429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553183-b371-44d6-bea7-76125ec69d88 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.168665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 196/399] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:50.191854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.205343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.206161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.206456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.206802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.216594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] RBAC: Adding query filter params (name=member) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:50.216594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.216594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.221468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 204/400] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:50.227984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.250247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.355827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.367780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcb4b9-7cd2-4d09-89c8-c56b297b8852 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.391715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72e4cf-4b8e-4165-800e-7f6a117aeaaf None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.420746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.421951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.422556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.422838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.425467 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.427169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ebf6c-a100-4bef-9463-ed4ed9d775a0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.429151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 205/401] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:50.430467 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4489e-0921-4e95-8374-24dbb677faa5 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.430753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4489e-0921-4e95-8374-24dbb677faa5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.431030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4489e-0921-4e95-8374-24dbb677faa5 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.432261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 206/402] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:50.433865 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.434678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.434963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.435240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.438193 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4489e-0921-4e95-8374-24dbb677faa5 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.454835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-86d974dc-d1c4-474c-8b3b-631f3c2827d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:50.466466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:50.467404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:50.468216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 197/403] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:50.469453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.469551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.469821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.495740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:50.497214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:50.543708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.571282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4489e-0921-4e95-8374-24dbb677faa5 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.581764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 198/404] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:50.583013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.628376 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.629683 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.630169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.630649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.632113 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.633812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.636205 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 207/405] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:50.641922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.675846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.714230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b42d-c13d-4491-afe6-befd51d83629 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.749664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60738f-f349-4d4c-80bf-16beb0667b70 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.750013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.750629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.750912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.751188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.751937 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:50.752855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.755047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.763205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 199/406] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:50.768472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.789586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.790740 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.791242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.791724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.803223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:50.811469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:50.812352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:50.813532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:50.816890 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 208/407] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:50.818079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.832062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.871772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28978f-ce1c-4272-b5be-06bc7b4cdc0d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:50.905156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:50.907833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:50.908918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:50.909380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:50.909669 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/groups/d9010b3c5d294d7e9574e812de5442f0/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:50.911889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, group_id=d9010b3c5d294d7e9574e812de5442f0, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:50.913444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:50.934137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 200/408] 10.222.0.21 () {58 vars in 1442 bytes} [Wed May 25 13:37:50 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/groups/d9010b3c5d294d7e9574e812de5442f0/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:50.935330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:50.950432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0377db5-af9f-48b9-8b45-47c4007968bd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.004030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.004948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.005255 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.005807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.007459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.009802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.012364 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 209/409] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:51.017717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.063796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf780ca-55b1-4a95-b449-e2511ca23da6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.104590 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.105639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.106115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.106537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.107468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:51.108003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.109743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.121430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 201/410] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:51.127585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.197796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:51.201109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:51.228331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04fc4c-3921-44ae-9b81-65270976f8f9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.271373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.272699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.273187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.273663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.276321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.278111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.280331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 210/411] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:51 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:51.287455 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.297035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d297c0-f4b5-4508-9f6e-e831882f35f1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.364788 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.365681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.365968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.366247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] PATH_INFO: `/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.367638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] RBAC: Authorizing `identity:get_group(group_id=5abb7ae1a7e04bbfbc618a909fdeeb1d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.369248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.375645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 202/412] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:51 2022] GET /identity/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:51.377895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.468161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c31e0-6d44-400d-8343-805438f46800 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.513070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fe928-890d-418f-9955-1add76602706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.514013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe928-890d-418f-9955-1add76602706 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.514300 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe928-890d-418f-9955-1add76602706 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.514571 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fe928-890d-418f-9955-1add76602706 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7/groups/d9010b3c5d294d7e9574e812de5442f0/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.516858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe928-890d-418f-9955-1add76602706 None admin] RBAC: Authorizing `identity:create_grant(project_id=8dc4238400014620b44e47d1b740b7b7, group_id=d9010b3c5d294d7e9574e812de5442f0, role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.519186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fe928-890d-418f-9955-1add76602706 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.536865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 211/413] 10.222.0.21 () {58 vars in 1442 bytes} [Wed May 25 13:37:51 2022] PUT /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7/groups/d9010b3c5d294d7e9574e812de5442f0/roles/13f5475a26344e9e816475b8539030d3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:51.577918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556787f-5bbf-4d30-9b92-47d14c2ad963 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.578255 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556787f-5bbf-4d30-9b92-47d14c2ad963 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.578495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556787f-5bbf-4d30-9b92-47d14c2ad963 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.581583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 212/414] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:51.594413 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.594747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.594975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.618547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:51.632948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:51.634173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:51.634704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b8520-b3fe-42e0-859a-cf81db32230a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.657065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:51.686791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.687684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.687967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.688311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] PATH_INFO: `/v3/projects/d26306eca6ff451eb60119f9e4cf5383` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.691712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] RBAC: Authorizing `identity:get_project(project_id=d26306eca6ff451eb60119f9e4cf5383)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.693242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.695054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 203/415] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:51 2022] GET /identity/v3/projects/d26306eca6ff451eb60119f9e4cf5383 => generated 344 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:51.700673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.781699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 213/416] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:51.810014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:51.870360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902337-7ea3-4444-b28e-c7ed38de9b11 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.937572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660838f-5c5e-4f81-b26a-4fd4ae7a4a72 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:51.945327 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.946639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.947128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.947611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] PATH_INFO: `/v3/projects/d26306eca6ff451eb60119f9e4cf5383/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.949997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] RBAC: Authorizing `identity:create_grant(project_id=d26306eca6ff451eb60119f9e4cf5383, group_id=5abb7ae1a7e04bbfbc618a909fdeeb1d, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:51.951798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:51.980598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:51.980598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:51.980598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:51.980598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:51.983372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:51.984364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 204/417] 10.222.0.21 () {58 vars in 1442 bytes} [Wed May 25 13:37:51 2022] PUT /identity/v3/projects/d26306eca6ff451eb60119f9e4cf5383/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:37:51.995067 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:52.004929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:52.005914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:52.007250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 214/418] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:52.012815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:52.085276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:52.173641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa93005-f5d5-4a01-8ec7-3112936aec69 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:52.212077 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:52.212819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.213193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.213472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.214219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:52.214526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:52.215918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:52.225051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 205/419] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:52.230703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:52.359487 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab11e8-4b12-4fca-bc09-71fec4e85c04 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:52.402959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:52.403616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.403902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.404178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.499926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:52.513650 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:52.515153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328f9ef-2403-4937-97da-cde8dc8d77a3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:52.660365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 215/420] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:37:52.665653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:52.670630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528aac4-7fb4-4b33-8fb9-ca060cab8f81 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.670926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528aac4-7fb4-4b33-8fb9-ca060cab8f81 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.671214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528aac4-7fb4-4b33-8fb9-ca060cab8f81 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.672604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 216/421] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:37:52.679992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.680290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.680677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.700471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:52.701829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:52.822874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 217/422] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:52.843549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366169f-bb62-4876-9a9c-c2b434f0467a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:52.852960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:52.853670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:52.891001 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:52.910883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:52.911674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:52.911960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:52.912241 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:52.914958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:52.916423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:52.919673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 206/423] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:52 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:52.926486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.027244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108cd8a1-2b42-4925-80ea-c62393976257 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.082435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3347f-08dd-487d-8ead-4927c5359bb5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.093323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.094176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.094463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.094743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.116230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:53.124454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:53.125360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:53.126310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 218/424] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:53.134581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.161961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.162777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.163064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.163349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.165272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:53.166753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:53.168403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 207/425] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:52 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:53.181591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.280934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:53.287959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:53.306927 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92105bf-4c21-4b8b-a328-1e48bc56312e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.351191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.351874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.352153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.352443 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.353235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:53.353546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:53.354931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:53.364138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 219/426] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:53.370725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.420323 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717ae29-a110-43fa-994a-82f8288d5290 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.507897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.508653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.508943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.509296 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.510144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=7ab3476e508d41e482b975ae625333e3) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:53.510461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:53.511906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:53.526572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 208/427] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 611 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:53.552345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d2384-cb88-4e78-859a-94d59384ccb5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.600142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:53.601046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:53.601442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:53.601828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:53.640446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:53.723395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:53.725021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:53.836180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 220/428] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:37:53.844123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:53.953806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b944494-7e6e-46bf-bb26-9fe15dde1d32 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:53.964765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.020671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.020671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.020671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.020671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] PATH_INFO: `/v3/projects/7ab3476e508d41e482b975ae625333e3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.021044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] RBAC: Authorizing `identity:get_project(project_id=7ab3476e508d41e482b975ae625333e3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.021044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95624fa5-3dd2-46ba-8b45-fd62dfaf5714 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:54.026298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 209/429] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:53 2022] GET /identity/v3/projects/7ab3476e508d41e482b975ae625333e3 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:54.030311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.062151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336ca24-07f1-4438-a5a4-dcff9319926f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.062467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336ca24-07f1-4438-a5a4-dcff9319926f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.062739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336ca24-07f1-4438-a5a4-dcff9319926f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.064208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 210/430] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:54.073413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.073707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.073989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.105420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.119511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:54.120764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:54.167379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e1ad0-0e95-4f29-afaf-e644638911e0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.212332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.213275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.213564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.213840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.217363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.220077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:54.223279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 221/431] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:54.230891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.249765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 211/432] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:54.276135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.365942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ac0c-8640-42fb-9e44-c8079d60d49e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.388735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.399982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ab9f2-ad46-4aee-959d-d5efceff77bb None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.405046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.419669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.420323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.420637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.420923 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.421851 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=7ab3476e508d41e482b975ae625333e3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:54.422073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.423576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fc5f5-d3cb-4962-b019-cdf219dca448 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:54.430715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 222/433] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=7ab3476e508d41e482b975ae625333e3&role.id=13f5475a26344e9e816475b8539030d3 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:54.442073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab386-e4cd-4d18-8643-d2cb7db92a8b None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.442364 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab386-e4cd-4d18-8643-d2cb7db92a8b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.444957 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbab386-e4cd-4d18-8643-d2cb7db92a8b None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.444957 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 223/434] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:54.450664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.451041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.451319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.458100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.458932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.459214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.459774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.477531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:54.483183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:54.484867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:54.486714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:54.487814 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 13:37:54.488961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:54.491561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 212/435] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:54.499951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.589680 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 224/436] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:37:54.612902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.640484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe6374-7f61-4937-9f16-a0bd452eac8a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.688870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.690396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.690798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.691170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.692034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] RBAC: Adding query filter params (name=member) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:54.692443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.694458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:54.706234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 213/437] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:54.712911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.760935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.763598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ff1b9-1975-4787-844b-595421cf6980 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.778564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:54.808459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.809342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.809624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.809945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.823239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:54.831522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:54.832473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:37:54.833342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 225/438] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:37:54.840350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:54.849875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54244bf-07a9-49ee-a987-e101245af0bb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.899447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:54.900498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:54.900830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:54.901112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:54.982510 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:54.982716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77b138-b28f-43be-914c-a4b1b6c6ffa8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:54.983977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7fa35-8f94-40ff-9b48-358428643e6f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.010699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.018710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.019383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.019672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.019971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.020735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:55.021049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.022431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670f177-3e12-4ad2-a1b4-3d8e6b654e86 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.030355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 226/439] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.031612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50e30-7d99-4484-925b-2ab7c44e5096 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.031905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50e30-7d99-4484-925b-2ab7c44e5096 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.032186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50e30-7d99-4484-925b-2ab7c44e5096 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.033979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 227/440] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:55.036740 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50e30-7d99-4484-925b-2ab7c44e5096 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.069057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.087865 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 214/441] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:55.089952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.090248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.090581 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.107061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:55.108266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:55.140751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50e30-7d99-4484-925b-2ab7c44e5096 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.175138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 215/442] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:55.175999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.176093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.176753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.177043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.177324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.230105 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.277492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c253663-3864-44b0-8cec-99b20f298869 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.278988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.280465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.311493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.312329 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.313210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.313493 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.313766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.315071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.316530 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.318286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 216/443] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.319052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.383060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 228/444] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 13:37:55.388150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.417924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209fba4-78bf-402f-a210-e28a98fa645b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.442265 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.453184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.453984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.454266 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.454541 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.465363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:37:55.473832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:55.474811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:55.475689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 217/445] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:55.476350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.492614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6db635-4989-4493-ab86-e86826759038 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.524628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.530188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.531056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.531340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.531620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.532843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.535760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.542644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 229/446] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.542644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.585660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d82a95-aefe-4674-a873-de7e625a5ffa None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.621655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.622520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.622803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.623086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] PATH_INFO: `/v3/projects/8dc4238400014620b44e47d1b740b7b7` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.624281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] RBAC: Authorizing `identity:get_project(project_id=8dc4238400014620b44e47d1b740b7b7)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.625769 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.627521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 218/447] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/projects/8dc4238400014620b44e47d1b740b7b7 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.628279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.646510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a9949-2b08-47ce-8407-ec575cc5e6e6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.680000 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.680628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.681321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.681598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.681959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.683357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:55.683667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.685112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.694715 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 230/448] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.695559 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.741058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24084f5a-056e-44cc-a9c4-2dba5a024083 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.763277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.787373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.788040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.788316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.788622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.789478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=ed8e20ce43ae47a49ef4e53d7cf3456d, scope.project.id=8dc4238400014620b44e47d1b740b7b7) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:55.789794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.791517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.799201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 219/449] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=ed8e20ce43ae47a49ef4e53d7cf3456d => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.800040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.815963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877893f-acc7-40b5-81eb-f90c1d7f4b0e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.861002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.861901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.862187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.862464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] PATH_INFO: `/v3/groups/d9010b3c5d294d7e9574e812de5442f0` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:55.863756 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] RBAC: Authorizing `identity:get_group(group_id=d9010b3c5d294d7e9574e812de5442f0)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:55.865305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:55.872190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 231/450] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/groups/d9010b3c5d294d7e9574e812de5442f0 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:55.873221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:55.930983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e5d16-9baa-4464-bdf4-66822a88e6e5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:55.952730 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:55.977157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:55.977890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:55.978111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:55.978392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.009574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491057cb-a9e1-4146-9ed9-6707c22ba04f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.049630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:56.056167 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.056981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.057194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.057434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.058283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] RBAC: Adding query filter params (group.id=d9010b3c5d294d7e9574e812de5442f0, role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=8dc4238400014620b44e47d1b740b7b7) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:56.058945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.060062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.061337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.062812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.067676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 232/451] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/role_assignments?group.id=d9010b3c5d294d7e9574e812de5442f0&scope.project.id=8dc4238400014620b44e47d1b740b7b7&role.id=13f5475a26344e9e816475b8539030d3 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:37:56.215785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 220/452] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:37:56.221578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:56.353209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b989f1-ed27-48ff-a12e-4ce830cc3c14 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.391549 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.392662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.393015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.393294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] PATH_INFO: `/v3/projects/d26306eca6ff451eb60119f9e4cf5383` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.394527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] RBAC: Authorizing `identity:get_project(project_id=d26306eca6ff451eb60119f9e4cf5383)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.395982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.397945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 233/453] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:37:56 2022] GET /identity/v3/projects/d26306eca6ff451eb60119f9e4cf5383 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:56.405405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:56.426065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:56.515114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6b5e0-48e2-4d6f-8c4f-6545f00aef44 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.551361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.552541 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.552867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.553137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] PATH_INFO: `/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.554318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] RBAC: Authorizing `identity:get_group(group_id=5abb7ae1a7e04bbfbc618a909fdeeb1d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.555768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a0564-9d81-437b-8b0b-70f9d3af1eb5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.557431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 221/454] 10.222.0.21 () {56 vars in 1180 bytes} [Wed May 25 13:37:56 2022] GET /identity/v3/groups/5abb7ae1a7e04bbfbc618a909fdeeb1d => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:56.563398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:56.639902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:56.685414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7cca51-557d-4171-b5a3-4f3e3cc3ba6f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:56.717488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:56.718140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:56.718430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:56.718712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:56.719538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] RBAC: Adding query filter params (group.id=5abb7ae1a7e04bbfbc618a909fdeeb1d, role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=d26306eca6ff451eb60119f9e4cf5383) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:37:56.719855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:56.722191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af5bb8-f16b-4b17-8c3d-406b912cc883 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:56.728111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 234/455] 10.222.0.21 () {56 vars in 1376 bytes} [Wed May 25 13:37:56 2022] GET /identity/v3/role_assignments?group.id=5abb7ae1a7e04bbfbc618a909fdeeb1d&scope.project.id=d26306eca6ff451eb60119f9e4cf5383&role.id=698416d22e5b4c278ff780c5eef36ede => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:37:59.497009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeae56-5757-4d6d-ac86-f6ae64f0ed71 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.498049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeae56-5757-4d6d-ac86-f6ae64f0ed71 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.498557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabeae56-5757-4d6d-ac86-f6ae64f0ed71 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.500379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 222/456] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:59.507380 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a81661-fc2f-43c8-9cb0-79296dc2fca2 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.508282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a81661-fc2f-43c8-9cb0-79296dc2fca2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.508795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a81661-fc2f-43c8-9cb0-79296dc2fca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.527219 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-83a81661-fc2f-43c8-9cb0-79296dc2fca2 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:59.528811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-83a81661-fc2f-43c8-9cb0-79296dc2fca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:59.631803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 235/457] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:37:59.672180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabeae56-5757-4d6d-ac86-f6ae64f0ed71 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:37:59.790852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.806887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9190a-5f53-4f5d-a56c-1a1e3f5fae38 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.807965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9190a-5f53-4f5d-a56c-1a1e3f5fae38 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.808743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9190a-5f53-4f5d-a56c-1a1e3f5fae38 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.810696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 236/458] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:59.817067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabeae56-5757-4d6d-ac86-f6ae64f0ed71 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:37:59.822478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ceeea-4878-495d-9054-594e2297fd9a None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.823006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ceeea-4878-495d-9054-594e2297fd9a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.823530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ceeea-4878-495d-9054-594e2297fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.852529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-445ceeea-4878-495d-9054-594e2297fd9a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:37:59.856171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-445ceeea-4878-495d-9054-594e2297fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:37:59.903440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:37:59.906126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.906838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.908825 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.931968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:37:59.934059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-617d8489-d3d0-49e1-a46a-3a5b8d1c97b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:37:59.967041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:37:59.967960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:37:59.968852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:37:59.968852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:37:59.968852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:37:59.968852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:37:59.969819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:37:59.971545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 223/459] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:37:59.976804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4eab-8edc-4862-83c2-86ea80dd8f8a None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:37:59.976804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4eab-8edc-4862-83c2-86ea80dd8f8a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:37:59.976804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f4eab-8edc-4862-83c2-86ea80dd8f8a None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:37:59.980857 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 224/460] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:37:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:37:59.981988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f4eab-8edc-4862-83c2-86ea80dd8f8a None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.011226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 237/461] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:00.016602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.016602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.016602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.049055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:00.051393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:00.144282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 238/462] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:00.149630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.189608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f4eab-8edc-4862-83c2-86ea80dd8f8a None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.232635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.233846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.234352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.234884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.236931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:00.237567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.239376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.242589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:00.259785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 225/463] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:00.261203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.282839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108f5cf-51d8-4aeb-9ef2-b241909f35eb None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.304050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:00.327447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.328279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.328891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.329199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.346856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.348755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:00.368404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:00.369359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:00.370294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:00.370294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:00.370294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:00.370294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:00.370294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 239/464] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:00.370855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.404218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f2117-268c-42f2-9b1c-03d9044804ff None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.457066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.458437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.458437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.458437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.472484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.474244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-99afd7cf-8e31-4db3-883f-009c276de720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:00.493580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:00.494757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:00.495884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:00.495884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:00.495884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:00.495884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:00.495884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 226/465] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:00.496170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450d092-9674-48cf-b110-882edc77d8ff None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.497299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450d092-9674-48cf-b110-882edc77d8ff None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.497299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450d092-9674-48cf-b110-882edc77d8ff None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.502283 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 227/466] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:00.503177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450d092-9674-48cf-b110-882edc77d8ff None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.513891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf5d9b-a7ee-4ae2-aeb4-c4d3ecd1b1cd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.562634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.563337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.563727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.564072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.565351 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.567347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.570875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:00.600620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 240/467] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:00 2022] POST /identity/v3/users => generated 246 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:00.601284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.650185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:00.663578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450d092-9674-48cf-b110-882edc77d8ff None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.733521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.735138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.736146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.736856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.738276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:00.739000 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.741886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7ddc7-3742-4d72-b3c7-67044d407354 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.769362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 228/468] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:00.771682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.772352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.772974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.804189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31db73-78ff-4c89-8d84-ce63f27df702 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:00.806101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:00.808924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:00.857106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:00.858318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:00.858318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:00.858574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:00.859326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:00.859641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:00.862100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:00.875472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 241/469] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:00.880275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.929906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 229/470] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:00.931217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:00.986545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:00.998431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:01.018184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c0e1d-0ad2-4035-9a53-3f2f55d8ef6f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.058545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.059222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.059512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.059790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.060512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.062013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce8fd8-2288-4af2-9aee-a48f1685b4ba None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.067024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba2b42-4685-4da1-bb47-f13d8fde4856 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.085242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 242/471] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:00 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:01.086512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf92f-340e-4f37-9597-51174c9e3f72 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.086807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf92f-340e-4f37-9597-51174c9e3f72 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.087085 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf92f-340e-4f37-9597-51174c9e3f72 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.088507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 243/472] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:01.093692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf92f-340e-4f37-9597-51174c9e3f72 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:01.128810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.130277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.130787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.131626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.133166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.135434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8480c-8225-40c0-b5d2-926422a3fb61 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.173120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 230/473] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:00 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:01.173120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.173120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.173120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.212632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:01.215144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:01.310218 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf92f-340e-4f37-9597-51174c9e3f72 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.340945 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 231/474] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:01.362157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:01.372030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.372893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.373178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.374029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.387969 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:01.395639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.398503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:01.399420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:01.429299 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:01.430206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:01.431127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:01.431127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:01.431127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:01.431127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:01.431127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 244/475] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:01.436824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:01.513557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f511e9-8951-42a3-87e0-a10fca9106cf None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.568023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.570012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.571144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.571739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.573821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.575609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.582841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650e29c-30e6-4b4b-8ec1-0d2aa8fdc64d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.593963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.sql.core [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:38:01.593963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 13:38:01.593963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [parameters: {'id': '79a12b963db14f75b6c267da6d921c91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 13:38:01.593963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91826) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application conn.query(q) May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result.read() May 25 13:38:01.632350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return ref.to_dict() May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:38:01.633333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application next(self.gen) May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application yield resource May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application next(self.gen) May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise self.value May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application session.commit() May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:38:01.634249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.session.flush() May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application compat.raise_( May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:38:01.635196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:38:01.636079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application conn.query(q) May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result.read() May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:38:01.638439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [parameters: {'id': '79a12b963db14f75b6c267da6d921c91', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:01.640415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 13:38:01.643133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.645186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 232/476] 10.222.0.21 () {60 vars in 1131 bytes} [Wed May 25 13:38:01 2022] POST /identity/v3/roles => generated 144 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:01.645186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:01.667125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.667803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.668088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.668366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.669128 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:01.669433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.670810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.698517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 245/477] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:01.704776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:01.806948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:01.864036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:01.876075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0e780-821c-42c1-b9db-627c1b1e3b34 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.892040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73b3f8-b0d0-4c6c-9bbd-567362600ac6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:01.945728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.946552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.946835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.947114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.956953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:01.957642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:01.957930 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:01.958209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:01.959197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:01.959840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:01.968210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:01.970537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 13:38:01.971739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:01.972493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5df91-a7ca-46c2-b7f9-616c8f77c852 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:01.972610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 233/478] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:01.982413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:02.010077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 246/479] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:01 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:38:02.169093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e7ea7-ea05-445f-85d3-fa6f75bf6bb1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:02.229504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:02.230275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:02.230561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:02.231491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:02.232344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:02.232688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:02.234866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc5c4c-2372-474c-b1c2-8eeadb311aaa None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:02.254387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 234/480] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:03.936860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc623f-6673-4a30-91bc-2203f8867666 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.937433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc623f-6673-4a30-91bc-2203f8867666 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.937433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc623f-6673-4a30-91bc-2203f8867666 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.939114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 247/481] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:03.944887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:03.945400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:03.945483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:03.966891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:03.968116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:03.975900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.007019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588756e-4a3b-49ae-8a63-bdba98138dbf None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.007361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588756e-4a3b-49ae-8a63-bdba98138dbf None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.007601 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588756e-4a3b-49ae-8a63-bdba98138dbf None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.010295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 248/482] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:04.014440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.014734 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.015014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.034149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:04.035376 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:04.088215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 235/483] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:04.133229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.166475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 249/484] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:04.193633 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.284656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55e2ec-04b2-46b1-8b8f-af60bee07484 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.284881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.315768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.316660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.316974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.317330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.327065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.333815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5bf2b50d-e6b4-4af3-a291-7fa820ef7267 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:04.343591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:04.345236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:04.346215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 236/485] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:04.347918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10de82-3121-4d2a-b73b-8dd4d5046486 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.348197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10de82-3121-4d2a-b73b-8dd4d5046486 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.348546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10de82-3121-4d2a-b73b-8dd4d5046486 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.354191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 237/486] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:04.354191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10de82-3121-4d2a-b73b-8dd4d5046486 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.382463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6392813-eec8-4b73-8e7f-2e2ef12c6b53 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.432972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.434387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.435792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.436613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.456340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-06787b5f-b237-40a8-994e-7fbfb3521ca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:04.468260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:04.469386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.470528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:04.477886 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 250/487] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:04.478854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10de82-3121-4d2a-b73b-8dd4d5046486 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.479846 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.480335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.480971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.499311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:04.501540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:04.517350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.518334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.518334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.518699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.519473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:04.519785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.521280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.530622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 238/488] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:04.531680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.599204 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 251/489] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:04.600468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.608595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.650418 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.652705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce24e2d-9b29-456c-a56f-fb7a15c06c9d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.691903 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.692747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.692930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.693217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.694003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:04.694316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.695847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.705971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 239/490] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:04.707040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.742651 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7df4e-46ca-419c-b712-b5b7100e6194 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.780651 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.782378 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.782900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.783464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.794886 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:04.796737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:04.817049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:04.817998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:04.818853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:04.818853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:04.818853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:04.818853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:04.819801 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ff87-4191-4ac4-b18a-dce67d1c3292 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.821494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 252/491] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:04.822643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.846709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.860838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.861770 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.862104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.862408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:04.874219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:04.882603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:04.883557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:04.884356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 240/492] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:04.885139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:04.932405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:04.954243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9486c7-258d-4627-a9e0-5bc9c248ef9a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.993098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:04.993814 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:04.994104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:04.994205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77c18f-7a81-4686-8034-4d75eb75e7ce None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:04.994667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.007900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.009377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:05.027111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:05.028046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 253/493] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:05.029726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.038230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.038911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.039203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.039480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.040223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.040535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.041973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ffaaa-5557-4d9c-acf8-2743223ca12f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.060503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 241/494] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.062147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f127d-c41a-487f-be00-8b45ddaa2728 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.062465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f127d-c41a-487f-be00-8b45ddaa2728 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.062718 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f127d-c41a-487f-be00-8b45ddaa2728 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.065238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 242/495] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:05.065898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f127d-c41a-487f-be00-8b45ddaa2728 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.118628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.146261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235e8f7-eb1c-4a07-8c76-2af0757383a6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.177731 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.177924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f127d-c41a-487f-be00-8b45ddaa2728 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.184720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.185393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.185676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.185953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.187158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.187466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.190149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.199434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 254/496] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.200029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.225529 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.226198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.226486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.226762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.227488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.227798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.229247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440fcc3-70ef-4058-93f1-38dd1e3b7ed5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.248759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 243/497] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.248759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.248759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.248759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.266140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:05.267438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:05.304582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e4c6d-4081-448a-a5dc-8f0619e428fd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.334649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 244/498] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:05.335511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.349963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.351325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.351832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.352336 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.366907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.369644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.371181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:05.387914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:05.389064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 255/499] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:05.389972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.442170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5e30e-1c04-4664-aada-522f5cc99c04 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.444891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.478310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.479071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.479356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.479627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.490178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.491653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.492575 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09809c32-1ba2-41c8-afef-74c91db24846 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-7c9dd36a-6e5f-4511-a525-e7b793b824ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:05.509621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:05.510475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:05.511338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:05.511338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:05.511338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:05.511338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:05.511338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 245/500] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:05.512048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2650a6-2c5a-409d-b9aa-bf0c8aab5848 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.512269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2650a6-2c5a-409d-b9aa-bf0c8aab5848 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.512549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2650a6-2c5a-409d-b9aa-bf0c8aab5848 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.515702 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 246/501] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:05.515702 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2650a6-2c5a-409d-b9aa-bf0c8aab5848 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.529406 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.530204 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.530490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.530767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.539003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.540472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:05.555215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:05.556122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:05.557163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 256/502] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:05.557727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.605478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.622607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2650a6-2c5a-409d-b9aa-bf0c8aab5848 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.657172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.657974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.658256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.658527 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.660393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b4706-9ee8-4f04-9836-67975d07daf9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.668641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:05.674996 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:05.677577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:05.678503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:05.679294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 247/503] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:05.679960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.700901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.701567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.701852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.702130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.702961 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.703269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.705183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e87a3-aedc-42ad-85dc-c9fd8a739558 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.743653 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 257/504] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.744995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.745290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.745572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.762668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:05.763789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:05.783379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac152138-e061-4924-931c-9ac5818d5920 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.816787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.817454 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.817741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.818018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.818736 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.819044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.820436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.830815 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 258/505] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:05.831612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.835165 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 248/506] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.836432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:05.847152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.941480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2acb6f-5e31-4814-af2b-793511904129 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.943271 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:05.944855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da742614-c499-4a01-8b77-fcd3c8b1d3d1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:05.978028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.978810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.979104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.979384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.980233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.980550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.981096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:05.981760 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:05.982024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.982123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:05.982300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:05.983042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:05.983344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:05.984774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:05.993542 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 249/507] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:05.994359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.000200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 259/508] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:06.000686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.067968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.108096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7248f94-314f-430e-83be-24d40307dc1d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.108349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912b4a4-5d13-42a3-b621-b3f3cafb0e1b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.143297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.143947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.144235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.144512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.145141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.145934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.146217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.146516 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.153623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.155129 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:06.169936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:06.170940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:06.171804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 260/509] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:06.172913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.197909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.217122 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.218584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.275769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371b4ee-391b-4028-9353-cf349a47b574 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.284021 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.309706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.310534 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.310816 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.311091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.318412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 250/510] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:06.319186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.321604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.323324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:06.340720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:06.341612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:06.342454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:06.342454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:06.342454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.342454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:06.342454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 261/511] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:06.342765 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.412075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.421849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623d589-8017-417d-8e4d-f643fbea3c6e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.442918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78632e51-cc10-4a9a-bf81-3380c38bed63 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.460918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.461709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.461996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.462269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.473147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.474609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.484406 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.485732 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.486021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.486298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:06.492860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:06.493759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:06.494606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:06.494606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:06.494606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.494606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:06.494606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 251/512] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:06.494879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.520034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.559761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.561387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.601729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953add7b-a3dd-4fa5-aaf9-cc53728b4d1f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.634118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.637006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.637693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.637979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.638260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.639456 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:06.639456 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.641734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.667049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 262/513] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:06.667908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.681536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 252/514] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:06.682257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.766442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.779182 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1034049-05dc-4c74-8b96-eac321670e4a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.789495 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547617fa-0ed4-4416-80db-d37502d0ff5a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.813159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.813966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.814260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.814536 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.823566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.824235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.824518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.824837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.825016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.825577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:06.825888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.826496 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.827298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.839864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 253/515] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:06.840966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:06.847468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:06.848385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:06.849239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:06.849239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:06.849239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:06.849239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:06.849239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 263/516] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:06.849505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:06.875479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:06.949717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c405-49e0-4750-a9c0-3fc25ffca418 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.952800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0686642-fcf0-4f99-ab1a-0a7517519f6c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:06.985001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.985685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.985938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.986213 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.986940 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:06.987255 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:06.988438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:06.989015 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:06.989382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:06.989677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:06.989959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:06.995994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.001030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.002379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.002694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 254/517] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:07.003704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:07.021365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:07.022278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 264/518] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:07.028087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.103481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.104441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162d21d-42ac-4c3e-874a-a8a7cad6ffc1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.129629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8a96c-41f5-42be-9e9a-27ea4f7c77af None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.142992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.143658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.143948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.144230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.167760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.168447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.168778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.169060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.169797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.171283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.194258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 265/519] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:07 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:07.195135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.232468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.238086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.239639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.346188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c09-aa5a-4e54-90bd-1fa8a9fb9e4c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.374767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 255/520] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:07.374767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.392878 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.404529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.405671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.406152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.406624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.407945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:07.408486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.411063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.427735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 266/521] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:07.428873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.515726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.521116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408f40d-2959-4e39-ae79-a10b8f08ac4a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.544331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101afb5-fd76-490a-b596-4de434c50601 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.561581 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.562425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.562737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.563017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.570937 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.572435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.581736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.582445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.582757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.586005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.586761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:07.587078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.588508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:07.589979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:07.590983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:07.591871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 256/522] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:07.593257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.609087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 267/523] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:07.610230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.671252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.715897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e71663-39eb-4f09-a02d-6a9ec147e5e3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.728220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148594d-afb2-4ad2-916f-5c0aa7e6dabb None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.755707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.756811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.757243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.757695 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.774431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.774856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.776165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.776354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.777010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.777621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.787314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.789298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:07.801764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:07.802796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:07.804310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:07.804310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:07.804310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:07.804310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:07.806152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:07.806991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:07.807857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 268/524] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:07.808838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.811578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 257/525] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:07.814104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:07.828195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.930520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ec1c9-6ffe-4166-97ba-7f55a34b3fb3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.936982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c3167-8a26-4741-9360-38ad2e2f5544 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:07.945954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:07.971003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.971724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.972014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.972297 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.973217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:07.973534 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:07.975011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:07.983546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:07.986509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:07.987062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:07.987503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:07.999784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 269/526] 10.222.0.21 () {56 vars in 1139 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.000361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.000750 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.000837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:08.016278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:08.017259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:08.018136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 258/527] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:08.018605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.111534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.143014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92218249-d23b-4dbf-8c3f-0820f220603b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.158551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b840d0-bd68-450f-9f7b-d29dbff8c590 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.201165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.202165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.202448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.202721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.203995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:08.204295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.205718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.207412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.208671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.209201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.209491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.210327 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:08.210645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.212131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.230585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 270/528] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.230585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.236602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 259/529] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.236602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.275999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.360116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c5806-8416-4cb7-9efb-5f232bdcb65c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.372728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add38a5-69ac-4489-b184-bda538c357f8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.392601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.403071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.404145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.404488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.404846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.406171 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:08.407277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.409545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.426350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.426917 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.427426 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.427566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.429921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 260/530] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.433681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.515217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.517180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.546031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.561635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434e859-7c32-415e-b805-0874137bfd9b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.604362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.605898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.606361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.607160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.615886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.617388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.632037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 271/531] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:08.632554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:08.633392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:08.634237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:08.635268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.638625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 261/532] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:08.639482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.691392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.746648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99c927-afa8-4458-a2fe-6efb320d32d0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.749517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4065c-7081-4ecd-a9e0-ebdfb53b9759 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.779525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:08.782554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.783441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.783881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.784159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.785026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:08.785338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.786761 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.786925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:08.787579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:08.787863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:08.788138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:08.788997 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=1f45bec6796b4e10a1b2a598e765fcce) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:08.789312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:08.790749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:08.804602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 262/533] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.804602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.816809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 272/534] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/role_assignments?user.id=1f45bec6796b4e10a1b2a598e765fcce&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:08.817820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:08.963533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe100c25-8cad-4279-8820-1140f080a5f2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:08.994182 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:09.039474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.039474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.039474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.040242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.061549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.064263 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.072112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db16571-26b2-4a45-a640-a90aae28fb6a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:09.087755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:09.088698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:09.089992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:09.089992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:09.089992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:09.089992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:09.089992 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 263/535] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:09.095008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.113767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.114801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.115360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.115936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.117075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:09.117591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.119489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.144903 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 273/536] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:09.146661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.208787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:09.258533 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b8991-4775-43ac-9742-e868c9a08ec1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.270571 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3ec43-29a7-42b4-a8ed-643487f63a61 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.300765 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.301766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.302258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.302741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.303785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=afc0939b6c0e4e6b81bc8337065cdc5d) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:09.304325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.306420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.319057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 274/537] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:09 2022] GET /identity/v3/role_assignments?user.id=afc0939b6c0e4e6b81bc8337065cdc5d&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:09.369439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.370132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.370421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.370701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.371446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:09.371758 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.373318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.395957 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 264/538] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:09.401547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.580287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f16a1-e794-498e-a955-9787aeb13297 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.616332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:09.643286 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:09.644435 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.644968 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.645417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.653282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:09.655079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-70969363-daaa-40f8-a5c0-930d75c034de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:09.671131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:09.672581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:09.673833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:09.675995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 275/539] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:09.683625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:09.723629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee365e-fa9b-46dd-a3b3-db52fa55d360 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.724201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee365e-fa9b-46dd-a3b3-db52fa55d360 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.724677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee365e-fa9b-46dd-a3b3-db52fa55d360 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.726560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 276/540] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:09.745176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:09.745470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:09.745758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:09.755045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:09.808003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:09.810122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:09.942120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f088f-c6fb-474b-8146-a61eac8686e2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:09.981630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 277/541] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:10.010417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:10.011385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:10.011812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:10.012228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:10.013206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:10.013683 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:10.015320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:10.034710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 265/542] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:10.041939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:10.042607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:10.151839 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:10.178116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594949c-cb1c-41f2-bec3-f2d40158758a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:10.298554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:10.299618 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:10.300129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:10.300679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:10.302412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=6d754f1173e446ceb607dfaeb2231c28) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:10.302982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:10.308593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:10.324599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 266/543] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:10 2022] GET /identity/v3/role_assignments?user.id=6d754f1173e446ceb607dfaeb2231c28&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:10.362294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474db66-18ef-4fc8-a6d5-a7ea7a75c87d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:10.383192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:10.433717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:10.435743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:10.436584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:10.437354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:10.461069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:10.477625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:10.479563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:10.481246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:10.487424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 278/544] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:10.494056 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:10.776371 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c20f1f-3b00-42ed-b036-3475bfa94979 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:10.876876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:10.877917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:10.878435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:10.879007 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:10.880514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:10.881104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:10.883757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:10.900838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 267/545] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:10.906860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:11.107563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b8e08-b8b1-486a-a9ff-5bfd6811cd38 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:11.168006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:11.169848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:11.170630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:11.171361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:11.193787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:11.197070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:11.232899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:11.234591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:11.236257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:11.236257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:11.236257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:11.236257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:11.236257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 279/546] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:11.242619 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:11.281883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:11.422877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f2ee6-1c63-48f8-aecb-4c688cd07e6b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:11.468592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:11.469677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:11.470249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:11.470842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:11.471799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:11.472305 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:11.473928 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:11.494645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 268/547] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:11.503145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:11.548620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:11.693213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e66f5-cd59-4f59-a835-d88a45e1eea9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:11.754006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:11.754833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:11.755118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:11.755396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:11.770021 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:11.771513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0a1c6457-59e4-40bb-8dfe-77c76323cbc7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:11.793201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:11.794677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:11.795742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:11.799107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 280/548] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:11.801739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:11.921247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:11.923779 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612736-cde6-4ca8-8e81-f5086909a0fb None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:11.924444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612736-cde6-4ca8-8e81-f5086909a0fb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:11.924967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6612736-cde6-4ca8-8e81-f5086909a0fb None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:11.927917 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 281/549] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:11.937675 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:11.937995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:11.938456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:11.974237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:11.975378 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:11.977880 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7125ef-ef34-4cc2-a896-8804ff93da4d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.053029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.053722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.054012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.054294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.055166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:12.055484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:12.056993 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:12.101810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 269/550] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:12.109787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.119225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 282/551] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:12.219637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.251075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:12.261823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630f83-0175-4514-a3a1-0b1e6531bf40 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.320164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:12.339272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.340347 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.340863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.341361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.392456 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecdca3-f9b3-4ba0-8e36-f69b33fdc58e None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.466864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.467967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.469445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.469950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.483158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:12.492880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:12.494012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:12.494880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:12.498510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 283/552] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:12.508612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.511549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:12.521365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:12.606353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:12.646985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb286f-c4c4-491a-8e5a-898c01563376 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.689130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.690235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.690745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.691228 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.692225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:12.692792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:12.694497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:12.708706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 284/553] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:12.711233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.738842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 270/554] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:12.744638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.795513 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:12.837161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8d14c-1db4-401b-8383-60008cca8404 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.873737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:12.881649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.882959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.883102 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ccd2-73e1-45f0-805e-474df5e2e788 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:12.883723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.884226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.899052 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:12.900606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b93b52a2-8921-4ca3-9388-995667d05598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:12.922998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:12.924689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:12.926323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:12.926323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:12.926323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:12.926323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:12.926323 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 285/555] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:12.927605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:12.927939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.928228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.928512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.930638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143b6a-00fe-4e4a-9598-6d64a1512346 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.931263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143b6a-00fe-4e4a-9598-6d64a1512346 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.931263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143b6a-00fe-4e4a-9598-6d64a1512346 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 286/556] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489944b9-2749-4257-adde-3dee36c596f1 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489944b9-2749-4257-adde-3dee36c596f1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489944b9-2749-4257-adde-3dee36c596f1 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 287/557] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:12.940617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489944b9-2749-4257-adde-3dee36c596f1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:12.947543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:12.949501 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-336340a9-acc9-4073-94d3-e1e2247389fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:12.977704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:12.979196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:12.980092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:12.980092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:12.980092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:12.980092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:12.980092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 271/558] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:12.980484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565b02-12f7-4298-8463-f2031c3a027c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:12.980987 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565b02-12f7-4298-8463-f2031c3a027c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:12.981413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565b02-12f7-4298-8463-f2031c3a027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.003326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-45565b02-12f7-4298-8463-f2031c3a027c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:13.004933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-45565b02-12f7-4298-8463-f2031c3a027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:13.101300 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489944b9-2749-4257-adde-3dee36c596f1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.125051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 272/559] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:13.125051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.125051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.125051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.154257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:13.156289 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:13.158550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.159232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.159515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.159802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.160538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.160873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.162277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.171996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.181143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 288/560] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:13.182227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.240051 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.276648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 273/561] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:13.276648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.353574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b89eb8-aa17-489e-8da3-ad9ca0f61d58 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.399696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762ef5d-2485-4e70-b254-8e99c141ab53 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.407378 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.408081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.408380 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.408708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.409434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.409749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.411185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.425763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 289/562] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:13.426638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.446731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.447755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.447888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.448104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.460837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:13.469614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.470767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:13.471738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:13.472705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 274/563] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.473744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.476755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.555045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d093bf-6c22-4eea-8866-f7fb95b22268 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.557706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.586147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805603c-3b44-4ea4-958b-d3163983c8fa None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.595535 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.596359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.596677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.596959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.605088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.606455 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:13.622028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.623117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:13.624076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 290/564] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.624807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.630298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.631118 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.631400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.631673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.643948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6c92f46b-5cc7-4f0e-9926-6019136a5b77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:13.651886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.662163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 275/565] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.663811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23379a-2d79-4a16-a228-4c6b5e3bfce1 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.664813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23379a-2d79-4a16-a228-4c6b5e3bfce1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.664813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23379a-2d79-4a16-a228-4c6b5e3bfce1 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.664813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 276/566] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:13.664813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23379a-2d79-4a16-a228-4c6b5e3bfce1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.732516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.750404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446379e8-004d-4e83-a4c0-ca432b66f2c4 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.781357 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23379a-2d79-4a16-a228-4c6b5e3bfce1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.789249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.790083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.790362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.790636 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.800096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.800367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:13.815598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:13.817066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 291/567] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:13.819341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.824815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.825512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.825796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.826074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.826851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.827172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.829411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.838949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 277/568] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:13.839826 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:13.853059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.933668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4698364b-265e-498d-aec6-23a071fb2dd3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.952074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4879-1291-4b4a-a426-5cfba53a670d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:13.976100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.976821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.977109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.977386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.978233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.978547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.979994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bb13-f7db-412e-9794-017ed2c109f6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:13.980795 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:13.993526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:13.994208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:13.994500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:13.994779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:13.995533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:13.995842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:13.997276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.003470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 292/569] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.004899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399061e-a984-4d46-a480-235962ee70ae None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.005187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399061e-a984-4d46-a480-235962ee70ae None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.005466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399061e-a984-4d46-a480-235962ee70ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.012898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 278/570] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.014611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.024625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e399061e-a984-4d46-a480-235962ee70ae None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:14.025866 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e399061e-a984-4d46-a480-235962ee70ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:14.103834 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 293/571] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:14.104528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399061e-a984-4d46-a480-235962ee70ae None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.121937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.146300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b56cc5-a3e3-4351-a518-e7158aede384 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.188075 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.188927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.189214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.189491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.190324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.190635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.192092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.205878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 279/572] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.206755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.240553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399061e-a984-4d46-a480-235962ee70ae None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.258209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.279628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.280627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.281049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.281637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.298403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.300803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:14.331696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:14.333112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:14.334075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:14.334075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:14.334075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.334075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:14.334297 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 294/573] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:14.335586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.336194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a6e44-e913-4c8c-80b2-53d3f797e8c9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.378616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.379434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.379720 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.379979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.392083 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.393614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.405503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:14.413416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:14.414417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:14.415300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:14.415300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:14.415300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.415300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:14.415300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 280/574] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:14.415613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.451451 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f48ce8-ae28-49f5-b9e9-e18cc6426711 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.492544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.493379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.493661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.493941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.506166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.507844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:14.526760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:14.527939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:14.528916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:14.528916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:14.528916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:14.528916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:14.528916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 295/575] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:14.529214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.530551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.535729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdad27-9aeb-4017-a62c-8a9b621ff86a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.573527 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.574333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.574615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.574889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.589689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:14.600160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:14.601155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:14.602005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 281/576] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:14.603042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.654474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1555cd5b-8ccc-4b29-ab3a-2fc896899765 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.673602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.694630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.695538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.695938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.696340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.697393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=17fa4d0ea2da404c91576db368cf28bd) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.697829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.699650 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.707230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 296/577] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/role_assignments?user.id=17fa4d0ea2da404c91576db368cf28bd&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.708261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.722934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d81be5-711a-40b2-9957-44f24b766019 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.775195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.776872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.777583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.778248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.780048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.780652 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.783157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.804061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 282/578] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.806015 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.838545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.850999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0741a4-75aa-46fd-ac0c-b67659308658 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:14.891033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:14.891712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:14.892000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:14.892753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:14.893182 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:14.893497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:14.894942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:14.909252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 297/579] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:14.910517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:14.970305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:14.974249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b64c77-b6a8-47d9-95fb-a797ce573b8e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.018319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.019103 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.019310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.019533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.020259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.020601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.021986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.034757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 283/580] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.035470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.055173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a5fbd-9f88-45b8-9644-fa9591c5a8a5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.097657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.098333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.098623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.099246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.099622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.099933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.101437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.115741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 298/581] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.115741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.153991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.199224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb2c893-061f-4cd7-a87f-5e2aecac34d9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.248017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.248802 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.249095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.249429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] PATH_INFO: `/v3/users/glance` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.260111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.261898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.274721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d62e9-a2c6-4e34-8ef5-1c22c3504469 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.276976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:15.283096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:15.285384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:15.287216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:15.289602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 284/582] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:15.291874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.324977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.325803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.326088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.326363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.334708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.336216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:15.351234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:15.352200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:15.353153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:15.360008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 299/583] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:15.360008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.453657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e770b98-af6b-4d53-9b4e-241de280393b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.501068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.501925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.502394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.502872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.505813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5613-86cc-42a7-ae8e-63b7abfc6bd6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.513320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.515935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.518862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:15.538754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:15.540776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 285/584] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:15.542676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.554650 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.555675 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.556090 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.557003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.572733 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.574572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:15.608368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:15.610252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 300/585] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:15.618019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.643296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.703005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ec083-7ab0-4b40-8d84-48a808260dc9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.754843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.755813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.756284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.756784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.758063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.758580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.761161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.788180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ba5a2-0e07-45c2-b408-ae1fd1edf581 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.796003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 286/586] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.796003 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.850531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.851572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.852092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.852541 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:15.853938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:15.854452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:15.857128 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:15.879382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 301/587] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:15.881884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:15.885394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.947932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94814156-987d-43e5-a9a1-7b8e176a9d6f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:15.996317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:15.997375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:15.998219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:15.998584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:15.999040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.000387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.000708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.002006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.022190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 287/588] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:16.022952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.040531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a16bb1-2f2f-471f-a594-39bb92707a14 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.096085 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.097267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.097996 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.098133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.099058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.099584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.102087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.121830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 302/589] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:16.130393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.191188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43926d-2b25-486a-bf26-dea1843dee34 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.223391 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.257858 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.259279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.259807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.260305 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.270574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761446e6-051f-4d7a-9710-6d87fcdcabf7 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.280840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.283578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.307546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.308440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.308760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.309040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/3df30cf4e410480a890d14264879d404` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.311567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:16.317113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:16.318036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:16.318917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:16.318917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:16.318917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:16.318917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:16.326568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=a21198d279bd499fbd1e2f42798f3918, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.326963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 288/590] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:16.329040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.329174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.345209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 303/591] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:16 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:16.346560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.497790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cc2c7-bb88-4ace-8e45-63e2bc80c1d7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.540840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290623-9a53-45a8-a884-41b403952da9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.544372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.545920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.545920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.545920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.558413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.559944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:16.583132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:16.584934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:16.586728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:16.586728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:16.586728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:16.586728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:16.588703 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 289/592] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:16.589644 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.616502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.625014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.625943 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.626233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.626513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.635037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.636755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:16.654190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:16.657058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 304/593] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:16.660353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.715212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.747068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80463-9e7b-4841-bc92-eb4691a9fcd5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.787192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97113c-ac49-453e-9e16-f07b688b39cc None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.798707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.799826 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.800121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.800862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.801921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.802246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.803953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.822061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 290/594] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:16.823521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.829116 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:16.830059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:16.830309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:16.830558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:16.831810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:16.831810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:16.833502 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:16.852674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 305/595] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:16.853763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:16.935276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:16.972393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc710-48b1-44e9-983c-ab50efa45f85 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.990265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ce60b-da7f-4656-9e63-a1c47fa9b897 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:16.994539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:17.025375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.026491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.027023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.027523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.028653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:17.029207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.031305 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.047254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.047254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.047254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.047254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.055834 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.056842 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.059494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 291/596] 10.222.0.21 () {56 vars in 1139 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:17.060993 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:17.080240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:17.081249 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:17.082149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:17.085119 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 306/597] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:17.086419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.196608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2396e6-42ed-4660-9472-db924b67a7b9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.231772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a7042-12ab-4b3e-b198-fca304a50d56 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.235899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.237512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.237512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.237512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.241106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:17.246109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.247845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:17.263520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:17.266191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 292/598] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:17.267984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.311748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.313215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.313720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.314209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.340865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.344357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2bfdfde9-298c-4cc2-97c4-0512ede2e9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:17.369584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:17.370558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:17.371452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:17.371452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:17.371452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:17.371452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:17.371452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 307/599] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:17.372462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3d93-fd3e-4a36-9e80-4f686cdd111b None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.372748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3d93-fd3e-4a36-9e80-4f686cdd111b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.372982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3d93-fd3e-4a36-9e80-4f686cdd111b None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.374438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 308/600] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:17.375009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3d93-fd3e-4a36-9e80-4f686cdd111b None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.398160 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:17.408468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b9c81-1d8c-40e2-a807-a3bc90f0e6fb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.464470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.465597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.466144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.466557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.467539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:17.468948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.470865 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe7a33-6009-4b28-83b5-611648bb5a55 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.488746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 293/601] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:17.491108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.491597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.492085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.513378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:17.514348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3d93-fd3e-4a36-9e80-4f686cdd111b None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.514707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:17.560025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.561274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.563255 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.563774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.565275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:17.565900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.569118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.591681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:17.598010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 309/602] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:17.598010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.615421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 294/603] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:17.616835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.708639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:17.747782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a3f9e-1a70-4398-8199-0e0e25200c64 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.768710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24278ace-cec7-4e79-89f9-6349003b92cd None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:17.800849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.801540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.801832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.802107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.802854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:17.803163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.804612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.816705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:17.817630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:17.817921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:17.818204 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/1f45bec6796b4e10a1b2a598e765fcce/roles/3df30cf4e410480a890d14264879d404` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:17.825074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 310/604] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:17.826338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.839927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=1f45bec6796b4e10a1b2a598e765fcce, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:17.842562 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:17.862638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 295/605] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:17 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/1f45bec6796b4e10a1b2a598e765fcce/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:17.864190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:17.924173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.022153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d0155-0a5b-4b33-9ce4-24548ff8e2c2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.024693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a1ec-ab9e-4844-95fc-b5f63e65a747 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.077497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.079252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.080319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.080809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/afc0939b6c0e4e6b81bc8337065cdc5d/roles/3df30cf4e410480a890d14264879d404` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.086143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.088376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.090093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.090797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.091300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.103946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=afc0939b6c0e4e6b81bc8337065cdc5d, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.106190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.114452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:18.125677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:18.126906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:18.127892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:18.131381 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 296/606] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:18.132511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.139781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 311/607] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:17 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/afc0939b6c0e4e6b81bc8337065cdc5d/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:18.156942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.299450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.323407 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7cf48-d22a-45ed-a480-21dc9559833f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.333695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f56dc0-cd7e-4f51-92db-b4b7672f3492 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.422631 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.424145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.424774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.425367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.437983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.440612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.441720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.441876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.442184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.442660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.443975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:18.444486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.448017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.463561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 312/608] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:18.473194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:18.474205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:18.475090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:18.475506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.476143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 297/609] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:18.483351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.587258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.630752 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861befe-3b0b-4dcf-a296-8bc7097fecd9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.700164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d70a18-f798-4c78-96f5-2daca3bd2a44 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:18.732882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.734228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.734803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.735306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.736912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:18.737466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.740253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.799253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 298/610] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:18.803280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:18.846751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:18.848073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:18.848612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:18.849127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:18.879494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:18.895425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:18.900590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:18.966775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:18.968337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:18.969281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:18.969281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:18.969281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:18.969281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:18.970096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 313/611] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 496 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:18.984924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.060284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2134ad3-e902-4b82-b888-4ff358cbc5a4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.113130 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.114623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.115058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.115494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/6d754f1173e446ceb607dfaeb2231c28/roles/3df30cf4e410480a890d14264879d404` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.142935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=6d754f1173e446ceb607dfaeb2231c28, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.145354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.168066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 299/612] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:18 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/6d754f1173e446ceb607dfaeb2231c28/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:19.266829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.274233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a2c73-be4e-4612-90b9-e09163aa0355 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.351309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.352937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.353727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.354474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.356206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:19.357158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.360308 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4317a-eaef-4ccb-ab7a-d363a1e0c3e9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.392790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 314/613] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:19.398386 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.531816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.584493 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f657-cd13-4ca5-80f2-a48cb8ae496c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.623568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.624437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.624672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46bbac-b1d0-4f97-a75e-6d364c47f5db None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.624835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.625045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46bbac-b1d0-4f97-a75e-6d364c47f5db None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.625209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.625358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46bbac-b1d0-4f97-a75e-6d364c47f5db None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.627184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 315/614] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:19.634010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:19.635572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:19.638248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.638575 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.638845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:19.652722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:19.655833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:19.664953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 300/615] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:19.666915 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.683265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:19.684462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:19.816184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 316/616] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:19.876964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:19.910369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecebad1-e774-49ab-bbaa-98d516fa09b3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:19.971076 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:19.993792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:19.995366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:19.996490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:19.998433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.001271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:20.003538 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.010792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.060082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 301/617] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:20.069833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.228975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.279824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc057796-b120-4d3f-93dc-895182ad989f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.349916 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03f40a-4e20-4c35-934a-1888e886cc95 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.352986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.353817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.354099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.355305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.369072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:20.378889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:20.381071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:20.382902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 317/618] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 507 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:20.391696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.464537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.464537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.467253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.467253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.489005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.492239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:20.566139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:20.567901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:20.569731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:20.569731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:20.569731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:20.569731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:20.570933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 302/619] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 502 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:20.578859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.678440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:20.791741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426a9a1-1bdd-4a4f-aba0-f91bf95fd55d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.843106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb0e7a-d804-477c-ad1f-37a2182dc5e3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:20.866065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.866778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.867117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.867431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.868200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:20.868511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.875389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.903360 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 303/620] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:20.907509 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:20.908332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:20.908712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:20.909062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:20.909932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:20.910315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:20.911848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:20.914483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.922256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 318/621] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:20.932343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:20.953837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.188718 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.199930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f1f51-295a-436b-9f3e-cdb32cce98d4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.213490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80917-fe32-4319-8e3f-6e2d166aada4 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.292630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.293966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.294474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.294980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.302124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.303442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.303938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.304444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.319473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.328109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.336630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.345138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:21.380418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:21.382228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:21.383978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:21.394049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 304/622] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:21.400239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:21.403640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:21.407166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:21.407166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:21.407166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:21.407166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:21.407166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 319/623] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:21.419372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.419763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.576214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.621925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b53eb6-3683-4b64-9f32-bc334e456fe2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.658589 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e81190-1dfe-4b47-ad76-4f6e40bee724 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.662206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.662882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.663178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.663515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.664258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:21.664599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.666145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d35d3-ec9f-4e58-8783-fa0389e5bf24 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.679902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 305/624] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.681365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d6f2-3906-477e-b4e4-067d5b2931b6 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.681660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d6f2-3906-477e-b4e4-067d5b2931b6 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.681947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1d6f2-3906-477e-b4e4-067d5b2931b6 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.687734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 306/625] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:21.689235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2de8d6-a7d2-434a-ad66-555a5bb1ff2c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.689533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2de8d6-a7d2-434a-ad66-555a5bb1ff2c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.689821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2de8d6-a7d2-434a-ad66-555a5bb1ff2c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.691187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 307/626] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:21.692281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5a4d6-ee78-4b1e-ad1a-bb257de2681b None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.692592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5a4d6-ee78-4b1e-ad1a-bb257de2681b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.692869 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5a4d6-ee78-4b1e-ad1a-bb257de2681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.712141 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-c1e5a4d6-ee78-4b1e-ad1a-bb257de2681b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:21.713391 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-c1e5a4d6-ee78-4b1e-ad1a-bb257de2681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:21.727732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.732748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.734116 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.734491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.734801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.735608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:21.735920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.737454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:21.757919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 320/627] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:21.757919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:21.816605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 308/628] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:21.816605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.816605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.816605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.845930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:21.848116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:21.935567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e971b9-f77c-4d92-8e19-2debbcfa3ef9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:21.963309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:21.977209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:21.978310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:21.978718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:21.979131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:21.989330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:21.991236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.008593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 309/629] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:22.008593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:22.016198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:22.018292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:22.020171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:22.025631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 321/630] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:22.027230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.152260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.184101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bbff1-a376-407f-bebc-7cab37bdb3cf None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.192168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c4c81-e2e0-44d1-a820-dc1de4fb80f9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.230435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.231466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.231757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.232033 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/17fa4d0ea2da404c91576db368cf28bd/roles/3df30cf4e410480a890d14264879d404` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.251230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.251581 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=17fa4d0ea2da404c91576db368cf28bd, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.252605 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.252775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.253077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.253529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.284605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 310/631] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:21 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/17fa4d0ea2da404c91576db368cf28bd/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:22.284605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.287308 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:22.302206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:22.304832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:22.307094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:22.312031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 322/632] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:22.312994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.403477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.435662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ec354-8dbe-49b2-9ecd-abe29845de6c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.482049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.492474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc725118-ab73-4b44-993f-e130564b051f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.497663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.498596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.499985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.500436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.501969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:22.502467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.505028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f32b60-d32b-41da-9f57-bdc53ac74f85 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.540659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 311/633] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:22.540659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9fce-fb6c-4228-b67c-3ed3f960367f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.540659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9fce-fb6c-4228-b67c-3ed3f960367f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.540659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f9fce-fb6c-4228-b67c-3ed3f960367f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.540659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 312/634] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:22.541390 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f9fce-fb6c-4228-b67c-3ed3f960367f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.551007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.551825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.552108 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.552385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.567867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:22.575797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:22.576931 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:22.578189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 323/635] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:22.578884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.712287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f9fce-fb6c-4228-b67c-3ed3f960367f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.754277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.755249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.755523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.756291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:22.756624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.758022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ce63-1f89-4aac-b122-e87bac2e2899 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:22.772613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 313/636] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:22.772613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.772613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.773072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558188f-b646-40e9-8a16-304b297f112c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:22.773240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.782502 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.795948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:22.798227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:22.821363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:22.822033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:22.822317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:22.822594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:22.866459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:22.908382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 314/637] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:22.911086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:22.943430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:22.946113 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.064164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95e61d-b130-41c7-a96b-783a2c618ddb None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.111405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 324/638] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:23.111405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.115661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.116910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.117819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.118252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.118673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.119727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:23.120029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.121593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.140608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 315/639] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:23.140608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.213454 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.286689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a5d44-fcfd-4f99-be86-3e82a5915f4d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.320462 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b965a3-e44a-4d0e-b8d1-1394f61c27de None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.330475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.331279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.331635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.331790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.345175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.346625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:23.366517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:23.368259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 325/640] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:23.370853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.387411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.388689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.389212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.389695 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.406442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:23.421035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:23.423128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 316/641] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:23.424942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.460865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.547177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39ee8-69ee-41dc-b99f-62530ab00e31 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.604883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.613108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a676e2-13a4-4ea5-b5d9-4d6d6c1a802a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.615690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.616624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.617193 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.617494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.632276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.633798 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:23.662686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:23.663942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:23.665164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:23.665164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:23.665164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.665164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:23.665164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 326/642] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:23.666775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.683403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.684209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.684938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.684938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.698982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.703332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:23.743628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:23.746317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:23.749641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:23.749641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:23.749641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:23.749641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:23.749641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 317/643] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:23.750543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.814267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69761f-edf6-4f51-8820-b23c46d58109 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:23.836387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:23.883923 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:23.884970 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:23.885458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:23.885945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:23.887233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:23.887702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:23.890569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:23.916261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 327/644] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:23.916261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:23.938655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b684a-ff37-40da-b31a-d9d6d34b50de None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.012173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.013394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.014484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.015178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.016657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:24.017227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.019866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.035824 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 318/645] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.037142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.067154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.113244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0856b71-de32-4143-af62-212a81aa6795 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.155106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.156049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.156445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.156831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.157619 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:24.157969 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.159708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.176478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 328/646] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.177700 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.229619 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.235359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323515-8eb3-484a-b51f-5668e3a53ac4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.294512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.299481 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.300030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.302334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.303607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:24.303607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.306475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.335143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 319/647] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:24.336597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.387334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73478512-06b7-4a9a-8601-f8ba43aae12f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.453896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.455137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.455639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.456112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.472074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.472683 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.474866 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:24.503594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:24.504994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 329/648] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:24.511283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.532054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f813687-6de3-4b2d-a17d-7f1987f9930d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.604389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.605776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.606254 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.606684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.627591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.630233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:24.662898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:24.663892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:24.665251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:24.665251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:24.665251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:24.665251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:24.668907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 320/649] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:24.669698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.671612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.688265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ff61b-d417-4942-8c62-512e14294d06 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.734611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.735709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.736272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.736728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.748962 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.751577 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:24.774933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:24.776729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:24.778503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 330/650] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:24.779462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:24.877836 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2777806-eac8-4f78-a7a9-527aa5ce870b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.928412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:24.934544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f62c8-1e32-430d-9bfe-3574243c1b22 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:24.950205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:24.951704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:24.952485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:24.953033 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:24.963150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:24.965113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:24.984119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:24.986299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:24.988157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 321/651] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:24.993042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.006552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.007889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.008499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.009168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.010865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.011563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.015145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.047992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 331/652] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.049438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.063801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.129011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03b66a-6ffa-4866-8d88-013ce6304840 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.162061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b972b1-3dfb-40d8-bc24-2087f516b34b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.166864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.167528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.167812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.168084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.168834 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.169141 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.170654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.183039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 322/653] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.183882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.202062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.202820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.203113 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.203393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.204198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.204506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.205996 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18915229-b315-4e83-b0d8-bdf1e2e801bf None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.219433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 332/654] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.226537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b974c9-cfa7-455e-ae02-09356b7b90d4 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.226537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b974c9-cfa7-455e-ae02-09356b7b90d4 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.226537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b974c9-cfa7-455e-ae02-09356b7b90d4 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.226537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 333/655] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:25.226537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b974c9-cfa7-455e-ae02-09356b7b90d4 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.246502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.296586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.305895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75122a7-f5bc-4f36-a5db-b2d3e07e3582 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.343770 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.344502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.344818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.344945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b974c9-cfa7-455e-ae02-09356b7b90d4 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.345119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.345963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.346275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.347712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.366081 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 323/656] 10.222.0.21 () {56 vars in 1137 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.366958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.393311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.393986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.394274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.394557 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.474637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.476128 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.484461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa434ba2-c027-4f5a-abf8-4b61177ed354 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.486235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.534397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.535568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.536063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.536546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.546317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.550067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.552728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c4a57d8b-8fb7-4701-a850-2f2df7f0aa03 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:25.582746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:25.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 13:38:25.585135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:25.585697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 334/657] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:25.585697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.585825 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 324/658] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:25.587582 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.588114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.588661 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.608709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:25.609900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:25.681777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 325/659] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:25.682384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.711698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6934405-6121-4d99-8adb-d7fb61552886 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.746657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.747306 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.747597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.747876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.748807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:25.749088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.750527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.756692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 335/660] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:25.757496 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:25.786146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.798861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9e649-e498-4c75-b679-5eb5eae7b360 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.806946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:25.846947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.847605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.847984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.848215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.894804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40294-7c3c-4c5a-bc61-4afa79eb6590 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:25.935328 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.936846 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:25.937292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:25.937583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:25.937827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.937945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:25.950982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:25.953474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:25.976778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:25.978682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 336/661] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:25.980472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.066521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.076710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 326/662] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:26.077359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.083510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.109288 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00767ba-d129-41d2-8011-6584c6f8a56d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.152375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.153087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.153379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.153679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.154582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:26.155139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.156635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.181482 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 337/663] 10.222.0.21 () {56 vars in 1139 bytes} [Wed May 25 13:38:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.181482 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.212093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f55f2-83c9-451a-8640-9657cbdaa827 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.254310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.255136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.255438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.255717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.270263 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:26.279192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:26.280917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 327/664] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:26.283193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.301397 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0488c-bf15-433e-aa55-69eb4afba012 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.352900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.353901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.354388 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.354861 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.356165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:26.356983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.359730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.360011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.376507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.385853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 338/665] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.386716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.440011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17458-9e0e-45df-99a4-4c0f2fa3fb5c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.485299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.486658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.487162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.487651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.506693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f056e-777c-4cb2-9d7e-bd5e7807957a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.509696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.511848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:26.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:26.547012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:26.548088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:26.548088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:26.548088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:26.548088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:26.548088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 328/666] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:26.549272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.564401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.565643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.566172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.566693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.667737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.671077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.673363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.798184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 339/667] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:26.799410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.871601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04504bad-4d98-4f65-a049-eb6da98537c2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.874705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.922849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:26.926342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f745f51-0cbe-4ec1-8b70-6f22273cfe01 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:26.930611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.931685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.932142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.932625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:26.933932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:26.934447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:26.937065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:26.955498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 329/668] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:26.955498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:26.993202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:26.994486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:26.994947 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:26.995400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.007296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.010084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:27.035645 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:27.036687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:27.037599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 340/669] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:27.038754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.130154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2a4f0-8519-457c-8ec7-f3e9e6a09293 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.190760 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.191744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.192228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.192636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.193901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.194423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.196885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.204337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff6943-a977-4ed3-9fa3-cd19a5217c8b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.221589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 330/670] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.221589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.256985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.270695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.272074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.272587 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.273168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.287608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.289144 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:27.309124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:27.310091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:27.311031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:27.311031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:27.311031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:27.311031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:27.311031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 341/671] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:27.312157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.330315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.374355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea1e53-d496-464c-b6ed-8fcff114a5ca None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.415147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.415997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.416279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.416547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.429880 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.431460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:27.451938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:27.453277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 331/672] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:27.455022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.462648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6dba3-8a64-4dce-bcf3-6f389966f431 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.508501 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.509634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.510217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.510717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.511761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.512166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.514944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.539442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 342/673] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.539442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.582039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.620585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bccce7-e51a-4380-a0d5-37214773526a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.647309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.662390 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.663340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.663621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.663904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.674849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.677684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.686478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c03b39a-2042-4007-92c0-eeabdd73e5cd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:27.693837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:27.702905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 332/674] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:27.708764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.737558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.739125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.739641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.740149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.741524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.742087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.744582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.759623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 343/675] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.761343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.868675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0299c-96e6-4a0d-8a2e-993bb95e56d4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.877770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a977e-eeea-4319-bc79-756ce7a06a21 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:27.917250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.925437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.925676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:27.926447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.926573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:27.926895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.927008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:27.927339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.927460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:27.928555 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.928808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=1f45bec6796b4e10a1b2a598e765fcce) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:27.929083 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.929309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:27.931410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.931639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:27.940538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 344/676] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/role_assignments?user.id=1f45bec6796b4e10a1b2a598e765fcce&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.941829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:27.946532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:27.958023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 333/677] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:27.958808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.110343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde12d99-9857-4108-9ada-fa15c4b6758a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.143811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5f3e5-29b8-4d08-964d-eb3a22173c13 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.167669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.168937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.169428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.170469 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.172003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:28.172591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.174992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.194837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 345/678] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:28.196210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.220212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.222061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.222629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.223186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.242382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.249666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:28.277396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:28.279575 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:28.281233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:28.288499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 334/679] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:28.288499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.305422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.395052 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810983b-6562-469d-8d9e-ca77f5578e27 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.416212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.453468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.454642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.455008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.455436 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.465702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.468181 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.475607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db462ce3-2a5f-4737-af8b-349d2be55581 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:28.489224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:28.491094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 346/680] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:28.493858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.524603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.525384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.525705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.525977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.526824 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=afc0939b6c0e4e6b81bc8337065cdc5d) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:28.527169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.528698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730df9b-e79e-44c3-8dd0-f7c0b0fb748a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.536621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 335/681] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/role_assignments?user.id=afc0939b6c0e4e6b81bc8337065cdc5d&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:28.537854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464916d-92cd-457a-b9ff-413c1923051c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.538146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464916d-92cd-457a-b9ff-413c1923051c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.538423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464916d-92cd-457a-b9ff-413c1923051c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.544848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 336/682] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:28.544848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464916d-92cd-457a-b9ff-413c1923051c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.695118 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464916d-92cd-457a-b9ff-413c1923051c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.730002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a79061-5418-489a-9fde-da3091a42f61 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:28.749178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.754648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:28.781205 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.781901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.782185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.782463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.783285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:28.783602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.784613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:28.785143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834e555-a5b1-493d-9c17-b85381b9176a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.786402 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.787660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.788976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.789869 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:28.790181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:28.791656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:28.806423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 347/683] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:28.807775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:28.808067 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:28.808348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:28.827125 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 337/684] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:28.829653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:28.838649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:28.839046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:28.938240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 348/685] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:28.939716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.015224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145993ed-3af6-4894-8fc8-e31c83dd4d7c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.075637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf2ada-68c4-474c-b97b-0a1a436bda68 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.078321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.079661 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.080354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.081069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.083168 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=6d754f1173e446ceb607dfaeb2231c28) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:29.083876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.086731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.099126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 338/686] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/role_assignments?user.id=6d754f1173e446ceb607dfaeb2231c28&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.101034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.102271 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.123619 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.124254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.124685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.125092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.208841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.210684 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.247413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.380774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 349/687] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:29.389508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.413040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.439214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e6aa7-1fa5-4568-aeae-424f42e2dd0c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.502890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.504780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.505505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.506245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.530680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:29.553465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:29.555250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:29.556800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:29.559710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 339/688] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:29.566269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.629294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963b19b-571e-4374-9493-b2ab145c4f89 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.668393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.668947 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.669242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.669561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.682619 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.684234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:29.705840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:29.707105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:29.708006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:29.708006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:29.708006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:29.708006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:29.708783 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 350/689] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:29.714602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.725863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b5964-12dc-46f0-bf07-e74eebf9e425 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.761199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.767479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.768515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.769041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.769519 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.770594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:29.771166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.772905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.780688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:29.785430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 340/690] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.795830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.839768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41701b9-767a-49bf-a21b-b8d8334da265 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:29.897964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:29.898639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:29.898926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:29.899263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:29.900042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:29.901549 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:29.902850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:29.932739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 351/691] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:29.939833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:29.942410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28cf18-4e2d-40da-90f2-6b58987e96b2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.008254 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.009085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.009367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.009641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.035549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.038907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:30.071980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:30.073052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:30.073941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:30.073941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:30.073941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:30.073941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:30.073941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 341/692] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:30.079667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.117173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.148084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed833e-876c-4b3f-a8af-8b492e13a291 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.188809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.237479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7403-44ba-4def-a082-1a9627f82a20 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.262003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.263594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.264148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.264725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.284178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.285208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.286146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.286633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.286738 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.287227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.288338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:30.288888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.290474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.313007 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 342/693] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:30.313337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:30.314210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:30.315278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:30.320595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 352/694] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:30.322962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.324589 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.574514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.634263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d885383-1668-4e1a-8006-2c1c3ab85299 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.638710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b5b78-fe40-45ff-b50e-7f9411b3c835 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.672435 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.673274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.673561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.673840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.682073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.683582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:30.698662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:30.699617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:30.701565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 353/695] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:30.706950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.743110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.744201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.744719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.745200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.746194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:30.746726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.748493 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.751548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:30.784589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 343/696] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:30.791166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:30.826177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3cda-50fb-428d-8517-8731a1b4ff4f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:30.866750 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:30.867828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:30.868325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:30.868905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:30.870009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:30.870565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:30.872275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:30.893674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 354/697] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:30.899400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.044925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.053542 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50f2f6-ac6f-4ddc-9d02-d7a778c98512 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.084048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568cb017-29a6-4eb1-8d8c-095699d42465 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.119447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.120230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.120554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.120859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.121698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=17fa4d0ea2da404c91576db368cf28bd) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:31.122108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.123558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.124256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4535af37-af6d-4586-95b7-2670221a84eb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.124925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.125563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.126264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.131028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 344/698] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/role_assignments?user.id=17fa4d0ea2da404c91576db368cf28bd&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:31.132411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe8cc-d867-4f13-9d40-5ad6df4e7294 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.132729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe8cc-d867-4f13-9d40-5ad6df4e7294 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.133012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4fe8cc-d867-4f13-9d40-5ad6df4e7294 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.139336 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 345/699] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:31.140860 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.141154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.141435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.146920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.176285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:31.176914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:31.278207 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.281642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.334621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 346/700] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:31.374084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.492747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890052e-e36e-4471-86c4-4f902ae9c511 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.525715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.532234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.533161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.533447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.533726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.546502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.548041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:31.571244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:31.573079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:31.575117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:31.575117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:31.575117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:31.575117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:31.575885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 347/701] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:31.579380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.685873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.694292 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f10cb-7f41-432b-a078-13dcdfc72a90 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.700763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 355/702] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:31.709830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.732707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.733304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.734772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.734772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.734772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:31.735169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.736386 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:31.751268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 348/703] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:31.757444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:31.818705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:31.904434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a005e1-19bd-475b-9f63-3df75d204539 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.922074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a060ae-bfb4-4b44-8047-80259354e626 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:31.981153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:31.982753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:31.983247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:31.983733 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:31.985032 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:31.987401 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.002429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.004481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.004820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.005288 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.021211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 349/704] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:38:31 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:32.023206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.025025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b512a8c8-495f-4789-9e1d-3be20d92c614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:32.047974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.049074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.050015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:32.050015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:32.050015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:32.050015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:32.050015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 356/705] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.055951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.151153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4f20-5c34-40d8-b100-8f2f1b98f2e2 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.151457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4f20-5c34-40d8-b100-8f2f1b98f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.151745 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a4f20-5c34-40d8-b100-8f2f1b98f2e2 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.153166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 357/706] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:32.166565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9ca6-a572-4fa1-8d96-639e2db3b24d None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.166812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9ca6-a572-4fa1-8d96-639e2db3b24d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.167095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde9ca6-a572-4fa1-8d96-639e2db3b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.192087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.198083 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbdc78-58bf-4013-a42e-c7e016fddb37 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.209870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0cde9ca6-a572-4fa1-8d96-639e2db3b24d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:32.211164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0cde9ca6-a572-4fa1-8d96-639e2db3b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:32.269867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.271057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.271816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.272661 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.274350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:32.275203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.278173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.302760 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 350/707] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:32.306008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.309482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.346063 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 358/708] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:32.353382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a096-2c81-4aa9-8e30-a533f9be6774 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.353772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a096-2c81-4aa9-8e30-a533f9be6774 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.353928 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a096-2c81-4aa9-8e30-a533f9be6774 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.355760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 359/709] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:32.363915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.364340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.364658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.389734 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:32.391018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:32.493774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 360/710] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 13:38:32.494544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.512722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd5e5aa-47bb-4a29-aeb9-a95047faa587 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.575145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.577193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.577969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.578756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.592789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.596014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:32.629548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.631598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 351/711] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.633339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.645826 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03245d8-b3dd-4a84-b44f-1a29af8159ff None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.654500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.658102 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:32.688287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.689175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.689465 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.689747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.690419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.692552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.711896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.712994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:32.713854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 361/712] 10.222.0.21 () {56 vars in 1129 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.715389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.788614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797e5e5-e1a5-4155-a758-690aff567a63 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.831399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b868088-c80c-4a4b-a4f8-891e74bf8019 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:32.846742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.848202 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.848863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.849345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] PATH_INFO: `/v3/services/network` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.850242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.852050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.871152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:32.871830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:32.872152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:32.872617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:32.873491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:32.873796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:32.874753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:32.875731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:32.876553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:32.877410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:32.880040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 352/713] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:32.881237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.892195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 362/714] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:32.893047 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:32.938204 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.024522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5160-179a-44c0-921b-ebe404baa1fd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.065057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.070345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e92c-73df-4889-a008-293ea89b29d2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.072138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.073259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.073710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.075172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.076896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] RBAC: Adding query filter params (name=network) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.077366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.079026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.089924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 363/715] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.090676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.131285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.132415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.132983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.133474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.134507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.135067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.136843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.143545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 353/716] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.145036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.242818 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:33.251165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8c12-89cc-410d-b943-8ec479b2d03f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.281672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c3504-26e9-4fd2-b10e-22eea7ff6346 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.292274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.293016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.293679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.293679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.294836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.295212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.296845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.304616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 364/717] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.305475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:33.370180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.370998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.371367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.371721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.373318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] RBAC: Adding query filter params (type=network) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.373745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.375583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c8d7e-f484-4ace-a334-99fb60867a30 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.388463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 354/718] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:33.444538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e39d0-7bb3-4bcb-af95-6fdfab68a94a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:33.492449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:33.493188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:33.493579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:33.493871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:33.494605 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:33.494922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:33.496342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ff3cd-c66d-4dc2-b5bc-fd3e84d0e102 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:33.506450 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 365/719] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:34.496280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363faf0-4d58-4354-a70a-b5b5bce753e2 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.496833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363faf0-4d58-4354-a70a-b5b5bce753e2 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.497374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363faf0-4d58-4354-a70a-b5b5bce753e2 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.502383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 355/720] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:34.505410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.505912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.505990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.534523 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:34.535737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:34.562679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:34.694490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 366/721] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:34.762179 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363faf0-4d58-4354-a70a-b5b5bce753e2 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:34.786899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:34.898977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363faf0-4d58-4354-a70a-b5b5bce753e2 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:34.939820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:34.940659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:34.941086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:34.941607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:34.964282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-398af04c-96c9-4ab8-9185-dedf4c05b762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:34.972174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:34.973785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:34.975820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 356/722] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:34.984632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.150359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619cbe3-e660-4789-a8de-6fe420ec2596 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:35.207887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:35.208990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.209480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.209956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.210910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:35.211440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:35.213075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:35.231011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 367/723] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:35.232624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255916-3376-4cbf-bb37-73dba01c6a6e None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.232934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255916-3376-4cbf-bb37-73dba01c6a6e None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.233249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255916-3376-4cbf-bb37-73dba01c6a6e None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.234651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 357/724] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:35.239643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.241733 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.242027 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.242313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.263707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:35.263707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:35.275661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:35.419276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a359ff-f00b-4c40-bd9d-07f39fb5b500 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:35.421801 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 358/725] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:35.461092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.463881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:35.492672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:35.493937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.494391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.494831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.534695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:35.538539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:35.571849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:35.572871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:35.573766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:35.573766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:35.573766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:35.573766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:35.581641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 368/726] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:35.602085 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.704771 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d656fc7-b7ac-47ae-af9d-b1c65fbd0a2f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:35.783815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:35.785078 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.785512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.785955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.800834 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:35.820955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:35.838499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:35.840782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:35.842972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 359/727] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:35.850707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:35.874942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25ec3a-6a89-4db8-b76a-4c0755ac7d1d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:35.940348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:35.941403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:35.941898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:35.942382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:35.943362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:35.943899 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:35.945566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:35.967008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:35.968410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 369/728] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:35.985764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.043566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf79573-043d-40da-85d7-84b68589ff3d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.115024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.120600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.140343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 360/729] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:36.147740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.220710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe743b-128e-4304-94a3-1a147ee3c20f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.254674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.268767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.270200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.270761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.271304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.284974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.287779 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.294525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843b551-95bd-47e6-8629-1f2f9a9e5c5a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-aac1fec5-485f-400f-9851-6112a6310770 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:36.317203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:36.318331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:36.319347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:36.321487 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 370/730] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:36.322798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.323574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662c82-4fba-409a-a5a1-5d425f26ed31 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.324140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662c82-4fba-409a-a5a1-5d425f26ed31 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.324470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662c82-4fba-409a-a5a1-5d425f26ed31 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.327880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 371/731] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:36.334139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f662c82-4fba-409a-a5a1-5d425f26ed31 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.342781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.343856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.344295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.344792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.361817 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.363601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5f9631fb-d997-408d-8611-c3bd991fd60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:36.389180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:36.390348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:36.391463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:36.391463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:36.391463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:36.391463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:36.394184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 361/732] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:36.395725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.396157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.396588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.426101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:36.427883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:36.477963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f662c82-4fba-409a-a5a1-5d425f26ed31 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.517386 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.518300 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.518755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.519137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.520714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:36.521155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.522829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.536910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 362/733] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:36.540643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.560951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 372/734] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:36.562756 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.633955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.691113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ae6e3-0cb6-4e81-bfc5-6272ccf899eb None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.695480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c504-4117-42d2-afe5-2aba455c5e9a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.701905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.741278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.742210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.742634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.742962 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.743574 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.744049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.744909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.745361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.745764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.745931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.747510 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:36.748006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.749858 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.763844 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 363/735] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:36.765134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.776653 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 373/736] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:38:36 2022] POST /identity/v3/services => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:36.776653 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:36.910430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf11b-585a-4433-9be9-f1a230aa8733 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.922078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4be9af-0697-4623-bff1-3bc4e54c2b3d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:36.924222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:36.953615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.954301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.954596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.954877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.968673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:36.969535 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:36.969829 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:36.970110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:36.978863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:36.980349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-adb1133d-d7f5-4ec8-9e51-dc352ef85a1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:36.998737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:36.999747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:37.000744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 374/737] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:37.006074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d68-31f4-406f-ad3b-a4cd7e3e58d6 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.006367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d68-31f4-406f-ad3b-a4cd7e3e58d6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.006649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d473d68-31f4-406f-ad3b-a4cd7e3e58d6 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.008159 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 375/738] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:37.009385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169254-54ad-4acd-a2bc-bc05f165138d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.009617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169254-54ad-4acd-a2bc-bc05f165138d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.009888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.010002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169254-54ad-4acd-a2bc-bc05f165138d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.011102 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 376/739] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:37.011821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169254-54ad-4acd-a2bc-bc05f165138d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.054180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.055646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377dd02-764f-4ed0-8b06-6830a074771c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.128264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169254-54ad-4acd-a2bc-bc05f165138d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.167269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.167955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.168244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.168616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.174111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 364/740] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:36 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:37.174281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:37.174281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.174443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.174443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.174443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.174629 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b928295-61bb-4fe1-90a4-7894b07b683a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.190557 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:37.191755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:37.193401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 377/741] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.194685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.194974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.195248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.212695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:37.213967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:37.229753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.266859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.275808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 365/742] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:37.276793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.300892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 378/743] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 13:38:37.301841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.396868 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b5b81-0d92-495d-b83b-248abf16b420 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.416553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f9d1f-1ef8-4829-92b5-7fb0afbb67d6 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.432964 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.433773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.434087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.434362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.445269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.446751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.456379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.457410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.457879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.458304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:37.466884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:37.468013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 366/744] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:37.468979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.486127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.533325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.546647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.548173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.606235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476c7fb-7186-4be7-9bfd-7a694c339ce2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.653239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.654059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.654345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.654626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.667972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 379/745] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 13:38:37.667972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.670497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:37.679593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:37.680553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:37.681450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 367/746] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:37.682137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.724397 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.777505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134af3c4-3e73-42b4-ae4a-e87d4668ae18 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.803920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eee9c4-d343-46f5-925b-b0d67eaf2a4b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:37.813057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.813659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:37.813775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.814009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.814296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.815025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.816401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.829722 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 380/747] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:38:37 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:37.830639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:37.860359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:37.861434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:37.861926 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:37.862301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:37.863133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:37.863517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:37.865088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:37.879669 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 368/748] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:37.880675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.002079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ca693-49f9-44fb-9f0e-7dc6dbd65301 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.004171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828ab48e-d5a5-4750-8606-a21cb131170e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.022060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.052106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.052783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.053072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.053348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.054157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.056186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.056186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.059656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.060470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.062077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.062077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.066775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 369/749] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.068086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.077373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.078859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:38.102174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:38.103127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:38.104010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:38.104010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:38.104010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:38.104010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:38.104010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 381/750] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:38.104761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.111359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.223391 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fe68-1bfd-46d3-b41a-4ca6b17a957a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.229466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6c3a6-ad7f-4750-a7e4-ecceda452e2d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.261161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.262124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.262635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.262982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.270722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.272337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.274881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.276112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.276403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.276710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:38.287668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:38.289478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:38.291861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:38.292917 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.293080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 370/751] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:38.293222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.293886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:38.314868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:38.316474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:38.317915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:38.317915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:38.317915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:38.317915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:38.317915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 382/752] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:38.322793 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.344714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.393677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.412610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453fb8f-0b7d-4f98-b796-4701bb2076a1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.464276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7f7c4-c330-4da1-a498-c71d7419cb2b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.466669 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.467586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.468086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.468632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.469607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.470126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.471730 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.485205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 371/753] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.486423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.508966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.509649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.509938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.510220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.511079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.511398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.512972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.536142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 383/754] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.537256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.605778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43d1ee-bce4-450e-9e2c-44a0b2df0e2a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.645335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.646660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.647664 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.648178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.648744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.649700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.650231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.651835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.665461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 372/755] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.666706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.676779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.677608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4166cc6-c212-433d-9bdd-139fee19446c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.718637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.719681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.719973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.720247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.728298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.729877 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:38.747949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:38.748980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 384/756] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:38.749971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.782658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9708a2-6a38-40d0-a640-cdfbc9778b02 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.829245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.830153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.830429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.831054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.832111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=17fa4d0ea2da404c91576db368cf28bd) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:38.832502 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.834175 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.848887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 373/757] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/role_assignments?user.id=17fa4d0ea2da404c91576db368cf28bd&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:38.849971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:38.885049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54e57c-64b0-4cdd-9355-511b2510f93c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:38.930870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:38.932017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:38.932515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:38.932965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:38.936753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:38.945522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:38.947508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:38.966717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:38.967749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:38.968724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:38.970029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 385/758] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:38.974933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.028977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.073782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102af77-7975-430f-88f6-2a939c000953 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.118816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.118816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.118816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.118816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.119525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:39.119847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.121755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.150072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 374/759] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:39.158320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.194602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1b37e-eb2a-4774-8e31-39dfbbaa3fbe None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.242827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.244009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.244313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.244626 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.245786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:39.246104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.249208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2c9e9-005f-420d-a8f0-a0858e404b48 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.272390 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 386/760] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:39.273759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3794-d047-4b27-af04-0016ec3e6523 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.274048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3794-d047-4b27-af04-0016ec3e6523 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.274326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a3794-d047-4b27-af04-0016ec3e6523 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.278142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 387/761] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:39.279005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a3794-d047-4b27-af04-0016ec3e6523 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.289395 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefbe6-9f05-45c1-b8e5-215593b91dce None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.328713 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.329421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.329715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.329994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.331093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=907ba93949a24ceeb776c213a75daac6) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:39.331413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.333018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab6bf1-0978-4436-868e-a0530465aed9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.341263 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.345026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 375/762] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/role_assignments?user.id=907ba93949a24ceeb776c213a75daac6&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:39.346580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.346876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.347161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.348191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.386560 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:39.390571 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:39.468019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a3794-d047-4b27-af04-0016ec3e6523 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.519465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 376/763] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:39.544135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.545676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.546172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.546707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.559938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.561994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.565665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:39.588674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:39.589658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:39.592237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:39.592237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:39.592237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:39.592237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:39.592237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 388/764] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:39.597037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.756393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a869fa-9d4f-4aa8-88fa-b3e37134a200 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.771420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e9b5a-79d7-4300-9aac-be26b88bc1fe None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:39.776922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.805326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:39.827659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.828470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.828781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.829063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] PATH_INFO: `/v3/services/network` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.829737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.832554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.836536 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:39.837243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:39.837527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:39.837847 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:39.838591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:39.838902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:39.840969 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:39.852951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:39.854773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:39.856772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 377/765] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:39.865392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.882069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 389/766] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:39.893655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:39.996234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162174c-d514-4f62-ad27-4723d50d9f27 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.045355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88767a07-0601-4971-af95-33cac3927984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.045953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767a07-0601-4971-af95-33cac3927984 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.046239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767a07-0601-4971-af95-33cac3927984 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.046517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88767a07-0601-4971-af95-33cac3927984 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.047239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767a07-0601-4971-af95-33cac3927984 None admin] RBAC: Adding query filter params (name=network) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.047552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767a07-0601-4971-af95-33cac3927984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.048973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88767a07-0601-4971-af95-33cac3927984 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.060006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 378/767] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.070004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88767a07-0601-4971-af95-33cac3927984 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.125127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd97eb3-fe48-4753-88d7-398d7b035ba6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.157674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.190018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.190832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.191118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.191399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.206251 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.207721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:40.244039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:40.246178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:40.249279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:40.259479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 390/768] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:40.267436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88767a07-0601-4971-af95-33cac3927984 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.270165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.304651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.307387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.307685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.307967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.310948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] RBAC: Adding query filter params (type=network) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.311280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.312725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.323284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 379/769] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.330326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.341702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.453688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b517769-0c0a-44a6-9616-0dd342ee5e53 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.457835 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce278b6-7a4f-48fa-b8d6-665f85df6cd0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.471554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.491029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.491790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.492153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.492508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.493405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] RBAC: Adding query filter params (interface=public, service_id=e797b34e9e514fb4a84e1fee6bc02560, region_id=RegionOne) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.493800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.495260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a43143-459a-4ef9-a3bf-259919ac0024 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.503839 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 380/770] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/endpoints?service_id=e797b34e9e514fb4a84e1fee6bc02560&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.505247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71131-4307-46b3-9b62-02488b31fef1 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.505616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71131-4307-46b3-9b62-02488b31fef1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.505972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71131-4307-46b3-9b62-02488b31fef1 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.507622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 381/771] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:40.514750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71131-4307-46b3-9b62-02488b31fef1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.530250 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.530922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.531213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.531497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.532291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:40.532635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.534639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59958726-e836-48ce-b651-931e34b235a5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.551098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 391/772] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.554752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.555221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.555733 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.576935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:40.578522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:40.676725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 392/773] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:40.677587 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.695424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.697812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71131-4307-46b3-9b62-02488b31fef1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.747504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.748379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.748796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.749185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.750106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.751717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.766049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 382/774] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:40.766290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:40.805395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de3b69-3e89-4cb3-82ee-d1a55b6da195 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:40.858898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:40.861613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:40.862599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:40.863857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:40.868274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=a21198d279bd499fbd1e2f42798f3918, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:40.868924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:40.872334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:40.903367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 393/775] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:40 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:40.981838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620c36-f284-4f25-aadb-77f7a69f8569 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.035470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.036415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.036724 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.037001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.037678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.039217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:41.061031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:41.062031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:41.062878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 383/776] 10.222.0.21 () {56 vars in 1129 bytes} [Wed May 25 13:38:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:41.068480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.163268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.193553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91dbf2f-f1ae-498c-9ade-e82bc2c13b8a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.232447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.233211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.233506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.233821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.234564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:41.234893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.236374 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2aaa7-ba34-4a35-b8f1-7b568551199f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.242104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 394/777] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:41.248577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.421682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885d2db-a3f6-43e6-8e78-d91288132719 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.438756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.454722 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ee0e3-f374-437e-a996-b2c10eaa7540 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.455060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ee0e3-f374-437e-a996-b2c10eaa7540 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.455303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ee0e3-f374-437e-a996-b2c10eaa7540 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.457002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 395/778] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:41.462664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.463243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.463762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.489609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:41.491550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:41.494520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.495469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.495926 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.496443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.499685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:41.500250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.501980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.516376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 384/779] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:41.521681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.642287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 396/780] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 25 13:38:41.676039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.682823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923698f-202a-4b62-83cb-24913b28219f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.726189 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.726881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.727171 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.727446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.728271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] RBAC: Adding query filter params (interface=public, service_id=235bca9a73f1494a90ef746ab76240bd, region_id=RegionOne) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:41.728594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:41.730028 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:41.744600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 385/781] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/endpoints?service_id=235bca9a73f1494a90ef746ab76240bd&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:41.744600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:41.810835 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.820620 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:41.875390 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e53ed-c32c-4407-86a9-4abebf360405 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.931445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98984ede-bff6-4959-a00d-a5f1cfe216cd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:41.945819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:41.946656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:41.946938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:41.947207 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:41.972150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:41.982018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:41.983014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:41.983870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:41.988232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 397/782] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:41.989540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.019614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.020754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.021249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.021727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.022622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.024274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62eb21-9042-478c-babc-96d56b7574a7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.034375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 386/783] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:42.123687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11356a27-aecb-44df-92b2-8fb0d3299dac None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.135803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d44d-1fd3-4aa1-89a1-f2358f8bb507 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.136150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d44d-1fd3-4aa1-89a1-f2358f8bb507 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.136398 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9d44d-1fd3-4aa1-89a1-f2358f8bb507 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.138545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 387/784] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:42.144250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.144549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.144868 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.165537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:42.166098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.167241 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:42.184681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.186036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.186503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.186946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.188182 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:42.191934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.198603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.231107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 398/785] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:42.237765 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.360192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 388/786] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:38:42.386378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.414578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3d5d1-4af9-40f3-8b2f-4f137d682782 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.421614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.461294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.462004 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.462293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.462568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.486512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.488203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.508731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783dc274-527b-4eac-ab14-834fedc39d2a None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:42.523078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:42.524036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:42.524966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:42.524966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:42.524966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:42.524966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:42.524966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 399/787] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:42.530402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.550281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.551462 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.551950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.552431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.570537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:42.583763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:42.584874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:42.585821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:42.595983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 389/788] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:42.608687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.641270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.758642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638f3a0-f2bf-4826-adf7-3c0d4ace8d3b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.812945 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:42.826976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed8f18-294a-4779-a99e-742c37ef71e3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:42.833284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.834001 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.834335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.834637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.835383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:42.835720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.837213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.863912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:42.864616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:42.864939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:42.865228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:42.865984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:42.866294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:42.867105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 400/789] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:42.867690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:42.873776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.878079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 390/790] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:42.883653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:42.989687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce7b1a-1761-4aeb-a70d-c25063564bc1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.034020 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.035619 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.036431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.037084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.053791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.055794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:43.077006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:43.078200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:43.079298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:43.079298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:43.079298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:43.079298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:43.081554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 391/791] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:43.089023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.105220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.162080 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db4f56-3b00-4f1d-9a7d-ac55b6211e63 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.165142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.247914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.248773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.249106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.249383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] PATH_INFO: `/v3/users/swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.262570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.264154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.267687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cef55c-b8fe-41b0-a8eb-a2576ab6300a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:43.287130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:43.288140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:43.289124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 401/792] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:43.297138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.319211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.320062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.320535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.320949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.321826 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:43.322283 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.323806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.338767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 392/793] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:43.345729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.477559 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.516279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b15352-a895-4998-9c57-a41738c069bb None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.539074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0229c-487b-4cee-9a52-1fea1805892b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.595520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.596277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.596576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.596857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.597725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:43.598038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.599551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.612799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.613728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.614013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.614288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.630100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.631619 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.642924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 402/794] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:43.648459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-99ae3aca-8d71-49d2-9da5-f8ea566c8e64 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:43.666593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:43.667604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:43.669129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:43.672820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 393/795] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:43.674416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a325c9a-36a9-42a5-a504-d857cbff9248 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.674728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a325c9a-36a9-42a5-a504-d857cbff9248 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.675005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a325c9a-36a9-42a5-a504-d857cbff9248 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.681286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 394/796] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:43.681286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a325c9a-36a9-42a5-a504-d857cbff9248 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.691171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.798989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2f29a-60e1-45d7-9c30-7bc6e627d579 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.807875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a325c9a-36a9-42a5-a504-d857cbff9248 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:43.837166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.840865 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.841860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.842146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.842475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.850458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:43.851420 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.851717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.852408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.853325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:43.853645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.854152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:43.855238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a532eb8-7f47-44a1-82dd-ca19bb549569 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.855836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:43.880044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 395/797] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0f8faa64-4a91-4e9c-99b1-c89dba04e936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:43.880281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:43.881124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:43.882084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:43.882084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:43.882084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:43.882084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:43.882301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.882301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.882301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.882504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 403/798] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:43.883971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296238ab-8e26-4db8-93c5-098cedebd079 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:43.884239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296238ab-8e26-4db8-93c5-098cedebd079 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:43.884573 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296238ab-8e26-4db8-93c5-098cedebd079 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:43.886150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 404/799] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:43.887325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296238ab-8e26-4db8-93c5-098cedebd079 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:43.904312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:43.906616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:43.984403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:43.993636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 396/800] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:43.993636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.024033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296238ab-8e26-4db8-93c5-098cedebd079 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.063318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.064191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.064483 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.064831 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.077622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.079270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5fd6b671-e899-4f0b-a660-03559027e425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:44.099372 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.100382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.101317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:44.101317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:44.101317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:44.101317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:44.101317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 405/801] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.102098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.102267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.102551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.119196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7f131-aa24-40fe-b8eb-f1937979ff17 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.120648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:44.123388 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:44.135788 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.160674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.161431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.161760 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.162076 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.162819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.163165 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.164644 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.177977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 397/802] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.178871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.211173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 406/803] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:44.212070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.278668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.317544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cfe5a-29f9-4a2e-9b8a-e926f62e732b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.332355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d678-32f0-4caf-b5c4-06dc17709278 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.359723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.360549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.360893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.361226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] PATH_INFO: `/v3/services/network` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.361896 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.363541 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.370637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.372291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.372807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.373294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.374221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.374772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.376358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.377700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.378793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 13:38:44.379710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:44.381036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 398/804] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.381194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.404974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 407/805] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.404974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.444183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.523647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45a4fd-4398-4e41-86e2-b41be552f4a1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.537019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ca91a-74ab-432e-bbee-d7a33fff6428 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.538680 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.563649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.564596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.564893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.565175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.577948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:44.579186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.580006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.580547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.580853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:44.588517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.589579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:44.590440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:44.591094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.591094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.592708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 408/806] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.593807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:44.609784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.610779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:44.611698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 399/807] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.612216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.722099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010df-7f47-45e1-a4a6-e5615c4b61b3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.729155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.736593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5029d2e-3ed7-4980-b4d1-44b6fe554936 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.763648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.764334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.764649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.764937 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.765678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] RBAC: Adding query filter params (name=network) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.765993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.767382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.777906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 409/808] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.777906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.781119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.781946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.782236 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.782521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.790450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.791974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:44.807526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:44.809294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 400/809] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:44.810344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.831870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:44.905481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48616fbf-ec2e-4145-9efc-b56503829842 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.923415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0ef56-bc9e-481a-9f6a-d2bc3ccf57ff None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:44.942412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.943089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.943375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.943698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.944458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.944798 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.946187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.955422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 410/810] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.956265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.958480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:44.959156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:44.959443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:44.959721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:44.960525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:44.960873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:44.962343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:44.974957 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 401/811] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:44.976073 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:44.996986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.064407 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.067807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d14789-cdf1-4059-ba4f-d1dbd4f40f9b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.081007 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6aa077-9be0-4613-a919-2408bedb1d34 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.107873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1f525-5712-4796-b586-3c300571975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.108885 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f525-5712-4796-b586-3c300571975e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.109423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f525-5712-4796-b586-3c300571975e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.109898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1f525-5712-4796-b586-3c300571975e None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.111108 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f525-5712-4796-b586-3c300571975e None admin] RBAC: Adding query filter params (type=network) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:45.111620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f525-5712-4796-b586-3c300571975e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.113152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1f525-5712-4796-b586-3c300571975e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.119188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 411/812] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.122027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ed7-06ab-43ac-8897-a7e6bfb0f911 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.122523 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ed7-06ab-43ac-8897-a7e6bfb0f911 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.123014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b506ed7-06ab-43ac-8897-a7e6bfb0f911 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.123437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.124103 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.124388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.124696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.128613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:45.128780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 412/813] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:45.128917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.129053 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506ed7-06ab-43ac-8897-a7e6bfb0f911 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.129205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.142995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 402/814] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.143824 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.227994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.234187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b506ed7-06ab-43ac-8897-a7e6bfb0f911 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.251867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e64301-5dee-439a-81e1-21cfa6f4e3b1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.269594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.270435 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.270718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.270991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.283070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.284719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.287392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.288290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.288596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.288884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/17fa4d0ea2da404c91576db368cf28bd/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.290923 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=17fa4d0ea2da404c91576db368cf28bd, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.292369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd87c8-125a-414c-a736-037bd5389736 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:45.303321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:45.304328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 413/815] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:45.305669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.311589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 403/816] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:45 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/17fa4d0ea2da404c91576db368cf28bd/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:45.313105 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.313410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.313694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.328551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.341908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:45.343221 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:45.432769 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 404/817] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:45.433780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.447013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab864708-d347-4f53-bb3e-7c13c0a9baa0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.489608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.490301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.490590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.491000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.491767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.493352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.510944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.523460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 414/818] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:38:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:45.524339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.563077 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f1fd4-eb90-48f9-ae41-a67ab21dfcb1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.617093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.637100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.637746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.638018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.638290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/907ba93949a24ceeb776c213a75daac6/roles/3df30cf4e410480a890d14264879d404` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.659634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=907ba93949a24ceeb776c213a75daac6, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.661603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.671476 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc153029-92dc-46e5-9cdb-359291ab4448 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.688228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 405/819] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:38:45 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/907ba93949a24ceeb776c213a75daac6/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:38:45.688228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.758755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.759462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.759800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.760278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.762257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:45.762588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.764391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.782780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 415/820] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:45.790295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:45.862978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5fd4c-4911-4669-a6d8-e221609efb98 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.881190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:45.943533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:45.944749 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:45.945192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:45.945625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:45.946750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:45.949848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:45.960031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb8cc2-96c4-4bfb-a807-4cbd9adfd826 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:45.966572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:45.968599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 13:38:45.970363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:45.972025 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 406/821] 10.222.0.21 () {56 vars in 1129 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:45.976094 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.026062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.028471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.028820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.029112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.040139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.041668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.059145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:46.065433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:46.067239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:38:46.069190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:46.076762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 416/822] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:46.076762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.103731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ab7b0-abff-4697-8043-3b9f6ae27ebf None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.155334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.156287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.156595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.156890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.157620 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:46.157935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.160385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.168696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 407/823] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:46.174321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.215169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f89b-15ac-4e2a-bc91-774d2f2a76e5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.229117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.271040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.271986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.272444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.273381 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.274817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:46.275297 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.277785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.302371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 417/824] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:46.304029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d09d0-0cd2-471c-98c6-1246e2962d1a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.308801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.348709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.349690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.350261 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.350717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.351602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:46.352066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.353699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.364772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 408/825] 10.222.0.21 () {56 vars in 1124 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:46.366346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.376975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.434959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8fab63-9740-44d9-b4a8-4c49ac2aaca5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.475897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.476639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.477026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.477564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.502159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f08a3-f685-4625-bf8c-87630c524b27 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.506088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.549679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.550939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.551452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.551994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.552989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.554763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.576776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.578259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.583353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 409/826] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:38:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:38:46.655266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:46.724002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 418/827] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:38:46.729705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:46.848505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c93017-4c6f-4666-822c-e2e552e6bf2a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:46.884799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:46.885604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:46.885893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:46.886308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:46.898738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:46.900288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:46.919349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:46.920376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:46.921332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:46.921332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:46.921332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:46.921332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:46.924672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 410/828] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:46.927088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.039655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cacff0-ad7f-4c73-bb11-deef3d362dae None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.077628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.078695 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.079195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.079697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.080936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.081459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.083060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.096585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 419/829] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:47.104630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.117014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.214928 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0ccc0-07a1-40fb-8fbc-e91e4cf120a9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.251662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.252474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.252789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.253066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.260444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.261998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:47.276607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:47.277907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:47.279686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 411/830] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:47.287458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.302366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.399394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7828e-a195-4bcb-a6c7-b76ea82c1048 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.435616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.436295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.436603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.436887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.437689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.438001 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.439449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b450e9-5dfc-485b-a2b9-131616618b34 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.453533 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 420/831] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:47.461645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.540952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.576984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efbd6-1327-466b-a442-04d819523cd8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:47.616242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:47.616954 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.617242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.617520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.618436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:47.618687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:47.620164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf969534-597e-4274-b7e2-6290ff77bc89 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:47.628045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 412/832] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:47.724933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51463d-ace2-4a89-ba95-0240d77e1b2d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.725297 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51463d-ace2-4a89-ba95-0240d77e1b2d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.725503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51463d-ace2-4a89-ba95-0240d77e1b2d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.727359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 421/833] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:47.734767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:47.735815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:47.736825 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:47.756447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:47.757701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:47.838415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 413/834] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:47.863184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51463d-ace2-4a89-ba95-0240d77e1b2d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:47.956547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:47.991626 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51463d-ace2-4a89-ba95-0240d77e1b2d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.029106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.029916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.030202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.030478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.041704 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:48.049460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:48.050474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:48.051538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 422/835] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:48.059252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.125772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:48.168095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed7a63-999d-4881-b5a5-9c773faa961d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.204192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.205163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.205545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.205822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.206577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:48.206883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:48.208279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:48.217656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 414/836] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:48.225760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.336505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240033-06b0-411e-98c2-39b8010eabc5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.368748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:48.375340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.376858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.377600 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.377876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.390925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:48.392427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-8843948d-c129-4847-9c1d-97ba6579fa63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:48.412365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:48.413331 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:48.414662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:48.414662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:48.414662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:48.414662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:48.414662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 423/837] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:48.421687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.533996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b0c8a-8ab2-4c3f-a623-ba3dad6c788d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.572650 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:48.576321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56710969-bc11-4384-8ee2-364a5455754c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.576642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56710969-bc11-4384-8ee2-364a5455754c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.576924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56710969-bc11-4384-8ee2-364a5455754c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.578490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 424/838] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:48.583100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.583789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.584087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.584374 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.584763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.585245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.585819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.586759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:48.587299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:48.588920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:48.603599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 415/839] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:48.607817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:48.609334 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:48.609646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.688612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 425/840] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:38:48.711171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.749290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075646a-2fc1-4706-9ec4-95f08b82d3c7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.791528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.793494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.794179 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.794635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] PATH_INFO: `/v3/users/swift` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.803459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:48.805528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:48.825511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:48.826514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 13:38:48.827525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:48.834925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9875a-9827-485a-83e9-e404c217dda8 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:48.835280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 416/841] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:48.837116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.847215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:48.872528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:48.873657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:48.873776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:48.874145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:48.887990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:48.896898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:48.897922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:38:48.899121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:48.903744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 426/842] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:48.904912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:48.906604 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:48.977442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b19e16-dc40-485b-a071-1a24b837daad None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.017547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.017780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9c4c1-202c-4d48-b4c5-cf3bb3e4734e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.018677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.019155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.019688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.020890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:49.021413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.023143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.051842 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 417/843] 10.222.0.21 () {56 vars in 1135 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:49.053900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.056678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.057432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.057986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.058682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.059367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:49.059848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.062531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.074226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 427/844] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:49.080747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.118466 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.179972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5f422-0212-483b-8ca3-83ae3440e571 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.181925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.195053 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1185c-be9b-4d23-9b05-400369439685 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.220956 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.221619 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.222043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.222217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.236737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.237635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.237922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.238598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.252680 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.254188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2b244eb3-1f9c-4c65-9cd4-a73f2dce3c41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:49.272588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:49.273785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:49.274855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:49.274855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:49.274855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:49.274855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:49.279201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 428/845] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:49.280721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6442109-7d76-45c8-895a-c8e4d658f8d2 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.281025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6442109-7d76-45c8-895a-c8e4d658f8d2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.281311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6442109-7d76-45c8-895a-c8e4d658f8d2 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.282927 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 429/846] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:49.283643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6442109-7d76-45c8-895a-c8e4d658f8d2 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.304555 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.306052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4ce9d-360b-43e3-ba82-7bc8cc5a354d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.394941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.427673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6442109-7d76-45c8-895a-c8e4d658f8d2 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.462600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 418/847] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:38:49.462600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.462600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.462600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.468422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.473074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.473748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.474028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.474298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.475024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:49.475329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.476719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.482960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:49.484520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:49.492985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 430/848] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:49.492985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.561873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 419/849] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:49.562984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.623781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a2ade-8855-49bb-94f6-f78cb44f936b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.668920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.670565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.671329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.671767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.684449 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e1142-d3ef-4cc3-bb46-6d44774cbb76 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.686357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.687854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.695963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:49.708741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:49.710319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 431/850] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:49.711759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.739638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:49.747119 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.747982 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.748262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.748545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.758297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.759820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:49.777324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:49.783621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 420/851] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:49.784634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.843131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4a555-eb7a-4cf3-a59b-df9b2d62e53f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.882366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.883187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.883476 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.883758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.884425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.885906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:49.899091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:49.900151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:49.902383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:49.903135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611472b8-66af-4264-a333-cf8fa826f578 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:49.905993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 432/852] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:49.906784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.950853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:49.951526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:49.951809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:49.952085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:49.952845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:49.953154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:49.954540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:49.968076 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 421/853] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:49.969437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:49.989584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.035552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.046315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cf5ab-ff72-4325-a15d-65469ea965bd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.091107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.091796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb9f90-6522-4b3e-8336-8605f2b536f7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.092238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.092767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.093273 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.094862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.095410 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.098193 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.120436 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 433/854] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.121396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.131113 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.131933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.132623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.132906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.133633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.133946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.135349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa117-a5c6-4b0c-a974-44ffcbae8320 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.140806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 422/855] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.142114 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4cbbb-4373-4769-ae45-d059835388c4 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.142405 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4cbbb-4373-4769-ae45-d059835388c4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.142684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4cbbb-4373-4769-ae45-d059835388c4 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.143943 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 423/856] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:50.144681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4cbbb-4373-4769-ae45-d059835388c4 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.233744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65185eb8-6383-4ad6-bc24-95cd686266ad None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.251395 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4cbbb-4373-4769-ae45-d059835388c4 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.254874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.269415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.270560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.271373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.271676 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.271959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.280385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.282188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.290144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.290812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.291097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.291376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:50.297267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:50.298327 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 434/857] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:50.299263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.364679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.366164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.413268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb7e42-3f6e-4f71-885f-3625b635958a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.447118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.447780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.448065 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.448884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.449989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.450298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.451677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76de225-c199-4036-8de1-4f94dd4689ab None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.456671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 435/858] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.457912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.458194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.458473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.478738 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:50.481247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:50.497384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 424/859] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:38:50.498465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.523804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.547957 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.598069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 436/860] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:50.599238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.687192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70174edc-0ed0-411b-af06-3fa3ed57e0a6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.730173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da53c6-97a9-4bf2-ae16-c05ba793bbf5 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.735330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.736412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.736926 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.737395 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.738534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:50.738859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.740861 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.767414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 425/861] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:50.767414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.778948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.781285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.781285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.781285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.793488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.794958 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:50.818786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:50.820696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:50.822760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:50.822760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:50.822760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:50.822760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:50.830365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 437/862] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:50.830365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:50.865699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.885239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:50.916259 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf744d-20f6-4fc1-a068-437033153f22 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.956860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd27cd-5d23-4de5-9c6b-c5743fcef9a2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:50.961305 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:50.962248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:50.962633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:50.963019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:50.963853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:50.965389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:50.979243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:50.980867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:50.989270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 426/863] 10.222.0.21 () {56 vars in 1132 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:50.990622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.011533 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.012298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.012732 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.013106 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.014050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=17fa4d0ea2da404c91576db368cf28bd) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.014583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.016753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.032913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 438/864] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/role_assignments?user.id=17fa4d0ea2da404c91576db368cf28bd&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.032913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.141290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11592921-2855-4cdf-9767-26c296816aa6 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.169900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.186278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a570ce-18a5-4fbc-9f82-3da4f4ce7d70 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.230005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.231884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.232474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.233080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.237256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.237862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.240748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.242769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.243712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.244071 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.244427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.246049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.246643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.248781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.255809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.259915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 439/865] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.260652 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 427/866] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.266232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.266971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.401043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf6a86-fdf8-4190-8398-2d800ed2376d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.420397 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7643-4385-4e24-aa59-64356563df0c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.440295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.441542 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.442087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.442566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.450016 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.451785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.463139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.464387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.464707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.464986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.465698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.466006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.467362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:51.469310 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:51.470186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:51.471082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:51.473897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 428/867] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:51.476739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 440/868] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:51.480055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.573709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.629836 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2ff9f-3952-4fbf-b6a4-853c9a823823 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.670909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.671999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.672495 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.673018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.674132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.674678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.676411 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814113c-a9fa-40a8-bc7e-e112906362ec None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.692753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 429/869] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:51.699524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:51.745335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:51.863955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7022f4-3f44-425f-bb1d-c3b07fba7723 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:51.925697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:51.927014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:51.927648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:51.928302 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:51.930094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=907ba93949a24ceeb776c213a75daac6) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:51.930795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:51.933757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5a1a-65f3-423c-96ae-bae4ca48f714 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:51.946076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 441/870] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:51 2022] GET /identity/v3/role_assignments?user.id=907ba93949a24ceeb776c213a75daac6&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:52.905596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8c45-b904-4dae-86a0-34f296f99a2c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.906109 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8c45-b904-4dae-86a0-34f296f99a2c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.906209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8c45-b904-4dae-86a0-34f296f99a2c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.907807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 430/871] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:52.913532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d7b7-08a1-4677-b87e-41bffb3c5662 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:52.913988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d7b7-08a1-4677-b87e-41bffb3c5662 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:52.914192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891d7b7-08a1-4677-b87e-41bffb3c5662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:52.934187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b891d7b7-08a1-4677-b87e-41bffb3c5662 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:52.935585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b891d7b7-08a1-4677-b87e-41bffb3c5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:53.023864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 442/872] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:53.052110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a8c45-b904-4dae-86a0-34f296f99a2c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.169125 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a8c45-b904-4dae-86a0-34f296f99a2c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:53.194152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.210973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:53.211649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.211921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.212199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.212967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:53.215110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71d90f-a832-43ed-8ca2-61b92ff2c10e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:53.233402 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 431/873] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:38:53 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:53.700471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e324a3-12d3-4a69-8926-fdff566a23fa None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.701056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e324a3-12d3-4a69-8926-fdff566a23fa None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.701511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e324a3-12d3-4a69-8926-fdff566a23fa None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.703285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 443/874] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:53.709480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d91c-2321-4a9f-9282-5f3d18d39739 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.709968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d91c-2321-4a9f-9282-5f3d18d39739 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.710395 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171d91c-2321-4a9f-9282-5f3d18d39739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.728793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9171d91c-2321-4a9f-9282-5f3d18d39739 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:53.732097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9171d91c-2321-4a9f-9282-5f3d18d39739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:53.842638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 432/875] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:53.873426 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e324a3-12d3-4a69-8926-fdff566a23fa None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:53.897913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:53.967375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31841eb-ed4a-424f-a2d1-80873797d155 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.967898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31841eb-ed4a-424f-a2d1-80873797d155 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.968017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31841eb-ed4a-424f-a2d1-80873797d155 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.969644 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 433/876] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:53.974934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:53.975228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:53.975517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:53.993517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:53.994889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:54.002202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e324a3-12d3-4a69-8926-fdff566a23fa None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.048827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.049649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.050094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.050377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.051341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.052807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:54.068803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:54.069860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:54.070709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 444/878] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:38:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:54.074038 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 434/878] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:38:54.081586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.093965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.133291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.175815 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.209266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbaa51-9f9c-490a-8f32-2f7380af1d2d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.221363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62150f-4c49-4a61-bf1e-661ca2d4c018 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.265043 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.265734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.266027 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.266309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.267048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.268467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.284127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.285092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.285518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.285932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.286777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:54.287242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.288785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a7175-9594-4ff3-bbb3-a5c2e3462fc2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.293052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 435/879] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:38:54 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:54.300304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 445/880] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.318034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.429891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08379-2b29-466a-bba8-62e47bc3ac16 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.430428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08379-2b29-466a-bba8-62e47bc3ac16 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.430922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad08379-2b29-466a-bba8-62e47bc3ac16 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.433269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 446/881] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:38:54.439811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:54.441603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.442140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.442637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.451446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ab91-0464-45c7-ae20-25cbceebd415 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.494009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:54.496275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:54.496422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332af7c-b334-407c-9677-652614c07761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.497100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332af7c-b334-407c-9677-652614c07761 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.497393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332af7c-b334-407c-9677-652614c07761 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.497679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332af7c-b334-407c-9677-652614c07761 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.498417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332af7c-b334-407c-9677-652614c07761 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:54.498733 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332af7c-b334-407c-9677-652614c07761 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:54.500531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332af7c-b334-407c-9677-652614c07761 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:54.512847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 436/882] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:54.607641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 447/883] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:54.634030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332af7c-b334-407c-9677-652614c07761 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.818956 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332af7c-b334-407c-9677-652614c07761 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:54.863696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:54.864923 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:54.865429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:54.865897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:54.879162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:38:54.887809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:54.888914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:54.889763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 437/884] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:54.895110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:54.969497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.001974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0a2d4-3203-4313-906f-70ff586a3b55 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.041837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.042652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.042911 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.043196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.043949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:55.044259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.045666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.054737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 448/885] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:38:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.061667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.083681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.175853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ea8c-5e2a-4b27-af9a-86f6c37cc022 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.214249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.215054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.215338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.215609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.226698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.228166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:55.246504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:55.247490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:55.248483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:55.248483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:55.248483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:55.248483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:55.248483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 438/886] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:55.254880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.371726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b106dba7-8bfe-4e0b-b992-80e3bb37d8ea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.408262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.411941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.412886 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.413369 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.413811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.414546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:55.414858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.416236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.430126 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 449/887] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.435947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.512488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:55.554221 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67086066-81f3-4f72-aa9f-661a37e1601c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.596679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.597532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.597816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.598095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.622362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.624339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:38:55.641287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:55.642312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:38:55.643285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:55.643820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 439/888] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:55.650270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.776846 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3952b-fe18-4d73-a68b-e18245625963 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.818422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:55.819111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.819398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.819679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.820523 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:55.820866 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:55.822328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b94ccc-3c27-43db-8d41-0f8b3d09ee50 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:55.842065 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 450/889] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:55.847925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:55.925683 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4751-767f-4fe5-9326-adb035ad06f3 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.926072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4751-767f-4fe5-9326-adb035ad06f3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.926296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c4751-767f-4fe5-9326-adb035ad06f3 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.928052 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 451/890] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:55.934156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:55.934463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:55.934723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:55.953282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:55.954867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:55.964963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30263b88-2165-4261-b4f5-061cc4f1dd61 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:55.996504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.006022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.006826 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.007335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.007735 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.017363 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.052926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 452/891] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:38:56.070491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.083378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.084884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.201883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 440/892] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:38:55 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:38:56.207429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.208655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d7791-6ac2-42ef-b1bd-e413c59a21de None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.245278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.253954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.255243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.255751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.256233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.257214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.259192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:56.281690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:56.283617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:56.285359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:38:56.296950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 453/893] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:56.301500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.333514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403db5b0-9daf-4115-89da-b96e9082e49c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.382044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.382845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.383123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.383397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.391480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.399244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.401170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:38:56.424977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:56.426782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:56.428655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:38:56.428655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:38:56.428655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:38:56.428655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:56.428655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 441/894] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:56.433317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.472939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29096-d65b-442e-9f6f-0ca111813d13 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.512015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.513138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.513625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.514117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.515036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:56.515596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.517213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.528625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 454/895] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:56.530073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.540671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.568496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27dff22-1717-4554-a6c6-c58598e6855b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.620046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.621341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.621875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.622351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.623384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:56.624643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.626794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.644942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 442/896] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:56.650436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.656742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2a08b-dac0-47e0-a0d6-9f9d7e2b8f20 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.669316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.707805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.708538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.708861 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.709147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.709896 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:56.710220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.711807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.725172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 455/897] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:56.726929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.797440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52166021-9791-480d-a5cb-84b053e70b44 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.835226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.836138 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.836434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.836747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.844688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.846227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.849101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.855661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8351024-56cc-44ba-8200-6cd18df75eea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:38:56.863545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:56.864511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:56.865453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 443/898] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:38:56.871373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.905626 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:56.907335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.908172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.908724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.910134 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] RBAC: Adding query filter params (interface=public, service_id=aa60946405c049fa9bf4cfd4d0cb47b9, region_id=RegionOne) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:56.911202 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:56.912733 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21038-5a57-4d90-a734-7830a89f9360 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:56.922921 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 456/899] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/endpoints?service_id=aa60946405c049fa9bf4cfd4d0cb47b9&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:56.924892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535698-6789-46be-98ba-822703dfcf4f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:56.926171 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535698-6789-46be-98ba-822703dfcf4f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:56.926686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d535698-6789-46be-98ba-822703dfcf4f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:56.929039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 457/900] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:56.930884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535698-6789-46be-98ba-822703dfcf4f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:56.975646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:56.992918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491ef6a-b89e-46ba-82db-54685b4c59b3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.034328 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.035001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.035302 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.035562 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.036359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.036704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.038167 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb6c72-9fb7-4a13-b64a-68abd7741410 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.055250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 444/901] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.055250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db90f7b-1cdb-4438-9639-464b60ef8ca7 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.055250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db90f7b-1cdb-4438-9639-464b60ef8ca7 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.055250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db90f7b-1cdb-4438-9639-464b60ef8ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.073761 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-8db90f7b-1cdb-4438-9639-464b60ef8ca7 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:57.075265 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-8db90f7b-1cdb-4438-9639-464b60ef8ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:57.083919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d535698-6789-46be-98ba-822703dfcf4f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.122832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.123604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.123962 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.124315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.125114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.126566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.133024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 458/902] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:38:56 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:38:57.133840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.147424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.173018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 445/903] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:57.174802 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f1a30-27b0-4bc6-991d-3dbc5538c3e5 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.174802 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f1a30-27b0-4bc6-991d-3dbc5538c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.176176 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f1a30-27b0-4bc6-991d-3dbc5538c3e5 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.184720 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 446/904] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:57.192244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.192506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.192950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.213019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:57.214357 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:57.295038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.298705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e81fd-e36a-47f8-b626-5b7e5ef8c7e2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.312649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 447/905] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:38:57.312649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.359751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.360989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.361464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.361935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.363444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] RBAC: Adding query filter params (role.id=ad75c144a36048c7bc05db8f7e59d70d, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=907ba93949a24ceeb776c213a75daac6) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.363976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.366583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.380767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 459/906] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/role_assignments?user.id=907ba93949a24ceeb776c213a75daac6&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=ad75c144a36048c7bc05db8f7e59d70d => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.380767 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.449296 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcaed3-9a69-495b-9edf-0d8bd4d3a257 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.473818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.490471 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.491306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.491596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.491877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.492583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.494059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:57.507900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:57.509035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:38:57.510634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:57.512418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 448/907] 10.222.0.21 () {56 vars in 1132 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:57.521885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.595064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f44146-1fad-4480-8a3d-f9522c10d459 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.643020 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.643994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.644287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.644584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.645556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.647581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.656098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2a020-eb6b-485f-ac3d-b50e75004642 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.667120 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 460/908] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:38:57 2022] POST /identity/v3/services => generated 241 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:38:57.720609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.721312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.721633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.721934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.722723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.723093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.724509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.731570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 449/909] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.738339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:57.793774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:57.895940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8597f40-d8e7-4c75-a096-f8274ed8124f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:57.936155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:57.937223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:57.937518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:57.937801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:57.938628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:57.938940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:57.940337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:57.947163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 461/910] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:38:57.953550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.009849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.078696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc49bc-e3ae-4191-8a64-56a2fa666580 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.121638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.122732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.123230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.123714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.124803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] RBAC: Adding query filter params (interface=public, service_id=07b042d1a22e4b3ea0baee7fd1e8a195, region_id=RegionOne) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:38:58.125377 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.127014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3d0c9-cd99-4bb7-9a9e-4b15a9c6a2ab None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.133895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 450/911] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:38:57 2022] GET /identity/v3/endpoints?service_id=07b042d1a22e4b3ea0baee7fd1e8a195&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:38:58.141769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:58.265303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:58.287348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1af6eb-3610-4615-9c5a-a23cb3c65470 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:58.332333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:58.333404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:58.333994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:58.334509 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:58.335433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:58.337226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72577245-4005-4c69-b4a4-8463bf5d150d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:58.345242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 462/912] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:38:58 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:38:59.611396 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d2034-f842-46b4-8dc3-ad29a33bd1bb None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.612245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d2034-f842-46b4-8dc3-ad29a33bd1bb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.612245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d2034-f842-46b4-8dc3-ad29a33bd1bb None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.613712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 451/913] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:59.619307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5345f-67c3-4436-a188-e88f85d0feca None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.620083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5345f-67c3-4436-a188-e88f85d0feca None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.620083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5345f-67c3-4436-a188-e88f85d0feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.639600 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-36a5345f-67c3-4436-a188-e88f85d0feca None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:59.640859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-36a5345f-67c3-4436-a188-e88f85d0feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:59.724274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 463/914] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:38:59.747707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d2034-f842-46b4-8dc3-ad29a33bd1bb None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:38:59.863587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d2034-f842-46b4-8dc3-ad29a33bd1bb None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:38:59.874323 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:38:59.895147 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3711-7c18-4336-af94-e76fbe136521 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.895769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3711-7c18-4336-af94-e76fbe136521 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.896253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3711-7c18-4336-af94-e76fbe136521 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.898192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 464/915] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:38:59.901942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:38:59.902847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.903139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.903423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.904110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:38:59.905016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:38:59.905510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:38:59.905615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:38:59.906094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:38:59.920060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:38:59.921346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 13:38:59.922180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:38:59.924924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 452/916] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:38:59.929632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:38:59.931062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:38:59.933181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.020652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 465/917] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:38:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:00.033775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.048051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929eb1-e632-4ad1-ab21-d90d3ea81fe1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.050216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.089269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.090361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.090850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.091331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.092267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.092838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.094491 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.100553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 453/918] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:38:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:00.108725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.165513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f3343-84c3-40fb-a9c2-1c2fe2a14395 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.178099 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.221123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.222804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.223278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.223718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.239977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:00.251740 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f17a6-48e2-4283-84e0-81b8c22bde93 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:00.252324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:00.253468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:00.254506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:00.258282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 466/919] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:00.263960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.298374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.299041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.299341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.299596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.300306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.300645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.302046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d7a30-83d1-41ed-a3c6-470575ad0d87 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.308517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 454/920] 10.222.0.21 () {56 vars in 1114 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:00.309403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97134e-b59c-45a2-b513-448b9776a0fa None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.309697 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97134e-b59c-45a2-b513-448b9776a0fa None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.309974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97134e-b59c-45a2-b513-448b9776a0fa None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.311425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 455/921] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:00.316084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97134e-b59c-45a2-b513-448b9776a0fa None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.351787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.386495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24ed94-a8e0-4f79-b49c-78965131c3bd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.426281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.428318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.429101 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.429385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.429741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.430520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.430742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.430990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97134e-b59c-45a2-b513-448b9776a0fa None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.432139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0d561-8c29-4a05-b3e4-7d50a0af7cfd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.444518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 467/922] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:00.445998 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.446427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.446779 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.465375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:00.466588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:00.474656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.475348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.475605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.475882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.476650 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.478034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.497959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 456/923] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:00.498927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.566753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 468/924] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:00.604616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.683856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.684773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda82497-1a21-47f9-b459-687e2e28758e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.735704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.736746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.737011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.737303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.743608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89de8b6-f8d2-45ff-be02-89c8310c2694 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.749473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.750972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:00.770927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:00.771892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:00.773573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:00.773573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:00.773573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:00.773573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:00.773573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 457/925] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:00.779144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.794473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.795431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.799964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.799964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.799964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.799964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.801606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-cc2910a5-9a75-4cae-9235-de4737665809 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:00.817616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:00.818748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:00.819581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 469/926] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:00.820395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51414a-c5f9-441d-a7a5-af6aed50d5e4 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.820395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51414a-c5f9-441d-a7a5-af6aed50d5e4 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.821111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51414a-c5f9-441d-a7a5-af6aed50d5e4 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.823702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 470/927] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:00.825110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51414a-c5f9-441d-a7a5-af6aed50d5e4 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:00.906852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a059ba1-d4e6-494f-9aa1-21ed39e1b48d None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.937341 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51414a-c5f9-441d-a7a5-af6aed50d5e4 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:00.948257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.948971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.949260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.949536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.950271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.950583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.951981 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e84d18-6b83-47d0-8567-c8fa4c1d252a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.963704 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:00.970277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 458/928] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:00.970277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.970277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.970277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.975292 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:00.975970 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:00.976240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:00.976518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:00.977284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:00.977588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:00.979743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:00.985066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 471/929] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:00.987625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:00.988871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:00.990310 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.058647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.068624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 459/930] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:01.068624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.103849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c8f9-7b16-412d-9528-49e9deec161d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.140216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.141079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.141628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.141628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.149362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.150855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:01.166560 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:01.167791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 472/931] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:39:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:01.168828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.187786 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d9dae-f138-4793-8da4-e698d424fdff None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.223006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.223678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.223967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.224252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.224995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.225309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.226692 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.229290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.232907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 460/932] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.233516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.287606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a0efa-f261-4240-9aad-bcb442278a0e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.300133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.325287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.326175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.326495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.326778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.327499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.329200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:01.342215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:01.343234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:01.344197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 473/933] 10.222.0.21 () {56 vars in 1132 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:01.345660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.349226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8caf212-07d1-4704-8c22-8bebc4221c7f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.387370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.388047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.388329 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.388624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.389464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.389881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.391340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.419500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 461/934] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.419500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.456628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9296aa-e9dd-4c5f-bc2b-e501ca7fe5d2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.469782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.493430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.494668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.494961 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.495242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.496133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] RBAC: Adding query filter params (interface=public, service_id=9e717179c0314f92b2a4fa9dbf1d6ffd, region_id=RegionOne) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.496454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.497870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.503073 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 474/935] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/endpoints?service_id=9e717179c0314f92b2a4fa9dbf1d6ffd&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.503875 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.533428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d0d51-b8cc-4954-9156-b157f3b6ad17 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.557582 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.572505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.573210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.573504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.573784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.574494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.574804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.576187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.581321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 462/936] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.582106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.623920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c682cd-8ea2-4ebf-8ed1-5275ff86c1f5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.659373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.660168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.660450 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.660752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.673019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.674602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.699239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6ed10-85f0-45a1-894b-567713e9b797 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:01.706854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:01.708017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 475/937] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:01.714554 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.759303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.759506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.760155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.760505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.761233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.762595 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.765017 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.775789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 463/938] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:01.777410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.848792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:01.882214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2e165-1480-4434-a9b5-4e8d6de33f82 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.916615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e6b48-703b-4cc6-a1a8-346a4bb0cc8e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:01.930717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.931841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.932364 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.932894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.933858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.934390 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.936095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.942539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 476/939] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.947768 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:01.967990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:01.968642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:01.968934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:01.969213 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:01.970256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:01.970568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:01.972305 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:01.988883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 464/940] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:01.995431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.068366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.082696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dab90-ae3f-4fcf-8a31-84ad8b950266 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.127138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.127879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.128169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.128496 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.129303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.130311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6345604-25cd-4aae-87fc-613ee1016439 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.130679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.144804 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:02.164597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 477/941] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:39:02.188160 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.189024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.189315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.189597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.197877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.199392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:02.216499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:02.217480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:02.218438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 465/942] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:02.224907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.350748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2259c0-a362-49b7-abce-eadc52475e55 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.393277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.394480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.395001 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.395282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.396164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:02.396475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.398102 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a915846-fb5f-4263-b1f6-2f2be4f5b632 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.416141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 478/943] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:39:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:02.423231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:02.540322 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0936e1-6c0a-476c-824b-17b702fc72db None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:02.593965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:02.594912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.595245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.595561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/907ba93949a24ceeb776c213a75daac6/roles/ad75c144a36048c7bc05db8f7e59d70d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.599264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=907ba93949a24ceeb776c213a75daac6, role_id=ad75c144a36048c7bc05db8f7e59d70d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:02.601400 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364fd62-a4af-4b0d-ac01-4795124670e6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:02.618159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 466/944] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:39:02 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/907ba93949a24ceeb776c213a75daac6/roles/ad75c144a36048c7bc05db8f7e59d70d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 13:39:02.892266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010017a3-cd05-41e1-a03b-973125149488 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.892814 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010017a3-cd05-41e1-a03b-973125149488 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.892897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010017a3-cd05-41e1-a03b-973125149488 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.894770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 479/945] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:02.901602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:02.902174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:02.902408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:02.927594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:02.929188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:02.953906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.031228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 467/946] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:03.055729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010017a3-cd05-41e1-a03b-973125149488 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.069472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.189355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010017a3-cd05-41e1-a03b-973125149488 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.238349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.239141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.239453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.239873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.240728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.242649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.244696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING keystone.server.flask.common [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 13:39:03.266975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.sql.core [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:39:03.266975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 13:39:03.266975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [parameters: {'id': '9b24870e67194c1e90562d82e12499da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 13:39:03.266975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91825) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b24870e67194c1e90562d82e12499da.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b24870e67194c1e90562d82e12499da. May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:03.288329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application result.read() May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 13:39:03.289424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application yield resource May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise self.value May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application session.commit() May 25 13:39:03.290837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self.session.flush() May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application compat.raise_( May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:39:03.291774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:39:03.293758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application result.read() May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:03.295644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [parameters: {'id': '9b24870e67194c1e90562d82e12499da', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:03.301040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:03.301869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9b24870e67194c1e90562d82e12499da. May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:03.302774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 480/947] 10.222.0.21 () {60 vars in 1140 bytes} [Wed May 25 13:39:03 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:03.303331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.447486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f1a23-4c5c-4e43-a069-416f424e26cd None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.492426 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.493428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.494244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.494482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.502104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.503911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f66c97bd-1096-4416-b449-f7e5e0a361e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:03.523815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:03.526284 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:03.528308 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 468/948] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:03.537082 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:03.688008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:03.743370 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4afdb-a496-483d-aa34-effd67d80165 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:03.783632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:03.784417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:03.784819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:03.785103 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:03.785959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:03.786269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:03.788059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da1718a-30cc-4fd1-842d-fd88f9b670c5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:03.811488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 481/949] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:04.628602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525cd0-8519-4df0-89c1-0602fc381d77 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:04.628602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525cd0-8519-4df0-89c1-0602fc381d77 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:04.628602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525cd0-8519-4df0-89c1-0602fc381d77 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:04.630662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 469/950] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:04.636949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:04.637986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:04.638485 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:04.660699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:04.663519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:04.702494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818768d-4db7-4841-811d-72fc3cfaba08 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:04.703039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818768d-4db7-4841-811d-72fc3cfaba08 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:04.703457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818768d-4db7-4841-811d-72fc3cfaba08 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:04.706080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 470/951] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:04.714627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:04.715129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:04.715999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:04.751961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:04.754663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:04.787046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:04.791096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 482/952] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:04.815022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:04.867049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 471/953] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:04.883034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:04.894942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:04.936906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99bd91-879a-4623-a1a6-373e028145d8 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:04.978012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:04.979434 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:04.979871 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:04.980306 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:04.981187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:04.982811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:04.997314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:04.998764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:04.999813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:05.002730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 483/954] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:05.009764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.047199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c40023-6e32-49b2-a4d9-bb2e34408169 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.097836 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.098768 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.099070 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.099351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.100307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.102156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:05.116633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:05.118071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:05.120458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:05.125174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 472/955] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:05.125174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.139994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:05.143585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc901d26-77bd-4ac9-a12d-c9be99428f86 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.164390 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:05.186184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.186983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.187377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.187849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.188792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:05.189198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.190823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.202055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 484/956] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:05.203206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.272452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5c809-8a53-44b7-8d2c-64ca36615a34 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.314099 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.314780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.315068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.315335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.316053 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:05.316361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.317785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105a757-7ed6-4c0c-9b12-1a383cc5975e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.323965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 473/957] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:05.330291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3f899-b7e9-4a79-91dd-b966f9d0f8bf None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.330591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3f899-b7e9-4a79-91dd-b966f9d0f8bf None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.330871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3f899-b7e9-4a79-91dd-b966f9d0f8bf None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.332262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 474/958] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:05.333025 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3f899-b7e9-4a79-91dd-b966f9d0f8bf None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.346419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95bb75-63d7-42ff-b28b-b77fcbacba67 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.388150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.388840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.389130 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.389407 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.390123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:05.390438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.391819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1553b-2265-45b8-98b9-c98851112526 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.404602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 485/959] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:05.404602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.404602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.404602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.419459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:05.422027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:05.443050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:05.484379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3f899-b7e9-4a79-91dd-b966f9d0f8bf None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.499083 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:05.536607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 486/960] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:05.548196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.549176 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.549644 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.550118 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.551048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:05.551523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.553170 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.560683 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 475/961] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:05.572647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.576865 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.716902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e873-7c4f-4f78-a8ff-b3227fd0e325 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.754322 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90e1ca-212e-4e5b-aa79-f022c8ea3f60 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:05.755954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.756661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.756956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.757238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.758013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:05.759681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:05.784300 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 487/962] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:05.827743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:05.838225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:05.839835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:05.841056 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:05.841470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:05.861516 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:05.871641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:05.872721 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:05.873566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:05.876858 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 476/963] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:05.880940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:05.938744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.037123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ac6588-67c7-49fb-b84a-ec72be6b3751 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.079346 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.080034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.080324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.080646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.081399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:06.081709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.083111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.092965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 488/964] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:39:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:06.100819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.234207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a647f3-6964-4b4a-843a-81ae69a17cea None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.287584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.288934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.289427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.289980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.309706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.312379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-fcb85233-1033-4dcf-8f84-0e85e5eb44f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:06.337039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:06.337999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:06.338900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:06.338900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:06.338900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:06.338900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.338900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 477/965] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:06.345679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.388772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fe525-1cff-4724-a872-e23cde381da1 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.389260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fe525-1cff-4724-a872-e23cde381da1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.389735 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fe525-1cff-4724-a872-e23cde381da1 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.398235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 478/966] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:06.401260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.401743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.402217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.429041 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:06.430556 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:06.452979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.460618 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06800252-6c02-4732-89e6-50c6f06e78b8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.486039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.526685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 479/967] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:06.540368 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.542379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.542825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.543309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.544196 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:06.544684 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.546233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.553588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.566422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 489/968] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:06.568045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.670132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f0a10-10f1-43c8-b609-81bbc9302319 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.712895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.716783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.716783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.716783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.716783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.717790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.719358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING keystone.server.flask.common [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 13:39:06.721719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6899ce-016b-4e29-9f96-b3b7703755a0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.738707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.sql.core [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:39:06.738707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 13:39:06.738707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [parameters: {'user_id': '31a1f411fe7d442490f66643c43b17be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 13:39:06.738707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91826) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result.read() May 25 13:39:06.753374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:06.754966 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application yield resource May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application next(self.gen) May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.session.rollback() May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.force_reraise() May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise self.value May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application session.commit() May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 13:39:06.756184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._prepare_impl() May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.session.flush() May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._flush(objects) May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application compat.raise_( May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application flush_context.execute() May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rec.execute(self) May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 13:39:06.757208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 13:39:06.758313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result = self._query(query) May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application conn.query(q) May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application result.read() May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 13:39:06.759726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [parameters: {'user_id': '31a1f411fe7d442490f66643c43b17be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:06.760615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 13:39:06.762110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:06.764708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 480/969] 10.222.0.21 () {60 vars in 1131 bytes} [Wed May 25 13:39:06 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:06.765705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.778785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.779959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.780435 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.780977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.790314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.793109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:06.812909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:06.814212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 13:39:06.816211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:06.820675 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 490/970] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:06.832157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.832395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:06.865663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:06.924340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa0616-112c-4811-9a38-aef5ac4b34a7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:06.968597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:06.969442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:06.969730 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:06.970010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:06.979467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:06.981421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:06.999434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:07.000790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 13:39:07.002087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:07.004111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c5f23-b3e7-4815-87eb-5019162143ce None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.005739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 481/971] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:07.006729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.057570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.059275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.059736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.060164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.061579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.062608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.064617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.099057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 491/972] 10.222.0.21 () {56 vars in 1149 bytes} [Wed May 25 13:39:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.102264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.131390 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.141195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882e01-52b4-4d47-8d59-c3c82c0cd360 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.185913 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.187430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.187918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.188673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.189694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.190219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.192630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.225030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.236484 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8268d-8d0b-4262-ad72-6f6265113989 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.258064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 482/973] 10.222.0.21 () {56 vars in 1101 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.289266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.290418 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.290942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.291394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.378719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.380646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.464338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.498228 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 492/974] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:07.503700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.624831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81cf86-db02-4f1f-8a50-4a6e5cf69931 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.681746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.683116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.683619 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.684122 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.703098 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.706101 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:07.741303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:07.742276 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:07.743172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:07.743172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:07.743172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:07.743172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:07.748363 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 483/975] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:07.749709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.865460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019652-7ce1-42b2-a800-00056803b36b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:07.905447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:07.906441 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.906889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.907339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.908258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:07.908770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:07.910406 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0fd4a-b000-47cc-848e-fa88ca3e4cc9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:07.924389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 493/976] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:07.929822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:07.948746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:07.988178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7a8a-523e-4aff-a0d5-a10693ea1394 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.988524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7a8a-523e-4aff-a0d5-a10693ea1394 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.988809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b7a8a-523e-4aff-a0d5-a10693ea1394 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:07.990245 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 494/977] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:07.996451 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:07.996780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:07.997081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.015805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:08.017076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:08.047503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.067912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5bfff3-86b5-4cda-9563-00ba27ba92b0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.098447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 495/978] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:08.107105 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.107930 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.108213 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.108490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.116469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.118564 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.123233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:08.134596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:08.135668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:08.136651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:08.140678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 484/979] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:08.141576 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.211900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.241869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5db0e-1d91-4d25-ac44-72fe709e4ab3 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.251024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ebff-2d3e-4139-aae2-7c77c737bbda None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.280049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.280757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.281046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.281328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.282051 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.283430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58738c63-1f0c-4d25-98d1-a1b70b939f36 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.285588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.286249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.286744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.286851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.287592 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:08.287908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.290674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.298414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 496/980] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:39:08 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:39:08.299723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68519f-5a65-4346-8d80-2c9ebfa29336 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.300006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68519f-5a65-4346-8d80-2c9ebfa29336 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.300290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68519f-5a65-4346-8d80-2c9ebfa29336 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.301820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 497/981] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:08.304092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 485/982] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:08.311108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.311709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.312025 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.312290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.339451 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:08.341821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:08.373096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.444613 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 498/983] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:08.454788 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc31d8-d24a-4a06-b4f7-f5c6831dd894 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.466145 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.486238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.497364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.497748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.498649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.498649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.499123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] RBAC: Adding query filter params (role.id=ad75c144a36048c7bc05db8f7e59d70d, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=907ba93949a24ceeb776c213a75daac6) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:08.499441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.500899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.507053 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 486/984] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/role_assignments?user.id=907ba93949a24ceeb776c213a75daac6&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=ad75c144a36048c7bc05db8f7e59d70d => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:08.588697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a5a6b-0470-4946-b612-9ed17a3a3dd1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.628684 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.629524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.629812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.630092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.630763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.632179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:08.645725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:08.646789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:08.647663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 499/985] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:08.655230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.767875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd558e6-e33b-493d-abbf-419b02579086 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:08.803747 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:08.804414 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:08.804737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:08.805026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:08.805785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:08.806100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:08.807498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:08.817303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 487/986] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:08.818646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:08.839479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:08.948908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ee690-bba9-4248-a5ef-770fe5fd6b2f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:09.011173 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:09.011860 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:09.012149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:09.012428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:09.013199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:09.013501 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:09.014902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:09.026146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 500/987] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:09.027241 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:09.087368 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:09.133312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897452b6-14c6-44cc-8aff-0745d97afc84 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:09.168195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:09.168883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:09.169175 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:09.169457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:09.170264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] RBAC: Adding query filter params (interface=admin, service_id=9e717179c0314f92b2a4fa9dbf1d6ffd, region_id=RegionOne) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:09.170583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:09.171972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc779f02-fee6-4c78-a571-be9b22f667b7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:09.180832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 488/988] 10.222.0.21 () {56 vars in 1249 bytes} [Wed May 25 13:39:09 2022] GET /identity/v3/endpoints?service_id=9e717179c0314f92b2a4fa9dbf1d6ffd&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:09.183275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:09.266740 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:09.291854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3fd33-e1af-4efb-b166-3e8f06c35e5c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:09.346930 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:09.347610 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:09.347898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:09.348175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:09.348901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:09.350699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccab4e-ab4e-4653-90a4-0afb64314f43 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:09.364844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 501/989] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:39:09 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:10.675963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61878-4297-43f7-8268-8f6a3bfd5725 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.676531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61878-4297-43f7-8268-8f6a3bfd5725 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.676531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61878-4297-43f7-8268-8f6a3bfd5725 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.677931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 489/990] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:10.684663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.685155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.685251 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.704023 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:10.705264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:10.775570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 502/991] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:10.798345 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e6966-449f-4cfc-9cff-80874be8c689 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.798627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e6966-449f-4cfc-9cff-80874be8c689 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.798912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e6966-449f-4cfc-9cff-80874be8c689 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.799013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:10.800409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 490/992] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:10.805410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.805828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.806117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.824492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:10.826054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:10.884399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:10.909971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 491/993] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:10.926876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45746658-6616-47fe-bd6a-aaf919fc5e0a None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:10.932407 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:10.968195 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:10.969349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:10.969349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:10.969624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:10.970236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:10.971612 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:10.984287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:10.990712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:10.991763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 503/994] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:10.993525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:10.994208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.050586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5980efc-17a3-48ed-91f3-216e6aa69266 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.087328 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.088184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.088467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.088772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] PATH_INFO: `/v3/services/image` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.089430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.090830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.103008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0493e-24d3-4698-be35-a53d5846c5f8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.103554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.104645 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.105500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 492/995] 10.222.0.21 () {56 vars in 1105 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:11.110609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.143999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.144954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.145244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.145582 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.146371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.146736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.148126 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.150020 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.154985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 504/996] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:11.160248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.223175 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3b4b1-42f4-424c-ba67-33c77c684f29 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.236908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.259573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.260246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.260537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.260867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.261585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] RBAC: Adding query filter params (name=image) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.261901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.263297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.268641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 493/997] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:11.274450 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d91a7-a83d-443b-93b7-29668fc4a0c5 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.274625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.311505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.312186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.312472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.312779 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.313491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] RBAC: Adding query filter params (type=compute) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.313807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.315176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.325683 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 505/998] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:11.328220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.389680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192cfb3-52f1-4e20-9c54-b2e525370890 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.399876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.427893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.428649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.428900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.429183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.429915 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] RBAC: Adding query filter params (type=image) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.430790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.432610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.438053 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 494/999] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:11.450747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157bc3f-40ee-4e42-9714-70c73ac945df None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.491412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.492186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.493232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.493592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.494489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] RBAC: Adding query filter params (interface=public, service_id=8b820491d32b4727b3efe9230b6a969d, region_id=RegionOne) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:11.494876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.496400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64552f4-f801-43b2-8efe-eaa5eb114ff4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.507290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 506/1000] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/endpoints?service_id=8b820491d32b4727b3efe9230b6a969d&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:11.509077 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.557355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.633278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876a06-441f-48ce-a874-fcadd02b525a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.635491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20d90-bb9b-476b-9151-3dfbfa256f60 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.636033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20d90-bb9b-476b-9151-3dfbfa256f60 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.636318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e20d90-bb9b-476b-9151-3dfbfa256f60 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.637736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 507/1001] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:11.643258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.643555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.643840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.662754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:11.663951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:11.671457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.672143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.672430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.672743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.673457 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.674832 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.682801 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 495/1002] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:11.747994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 508/1003] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:11.777762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:11.936533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae78ef-a8c2-4ca1-9a45-024730457910 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:11.939116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:11.976173 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:11.977059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:11.977343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:11.977621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:11.978311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:11.979772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:11.993763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:11.994960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:11.995823 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 496/1004] 10.222.0.21 () {56 vars in 1126 bytes} [Wed May 25 13:39:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:12.001230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.085044 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.135003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f083c8-d986-406c-b75c-f25859684f45 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.171178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.171855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.172144 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.172425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.173178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.173490 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.174867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.184746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 509/1005] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.186068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.295235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e9b0-ac49-4485-88ad-096db1bed8ce None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.330475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.331142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.331423 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.331700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.332404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:12.332754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.334120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016eac5a-8153-4a17-ac73-080e594ad5a3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.340158 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 497/1006] 10.222.0.21 () {56 vars in 1122 bytes} [Wed May 25 13:39:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:12.345545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:12.446874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f243486-cbc8-4cb9-8fd3-21e5e5782a92 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:12.468088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:12.484066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:12.484862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:12.485213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:12.485555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:12.486369 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:12.487795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d438c2-bfe7-4079-b1b9-dfc4cfa41e66 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:12.504549 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 510/1007] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:13.773381 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a42-4d7b-4c2f-a9da-3ac4d3d47d42 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.774719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a42-4d7b-4c2f-a9da-3ac4d3d47d42 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.774719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d8a42-4d7b-4c2f-a9da-3ac4d3d47d42 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.775991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 498/1008] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:13.781347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e94d1-0a76-4828-8df0-872f6da4877b None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:13.781803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e94d1-0a76-4828-8df0-872f6da4877b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:13.781897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e94d1-0a76-4828-8df0-872f6da4877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:13.798728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-e21e94d1-0a76-4828-8df0-872f6da4877b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:13.799919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-e21e94d1-0a76-4828-8df0-872f6da4877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:13.882423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 511/1009] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:13 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:13.906430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d8a42-4d7b-4c2f-a9da-3ac4d3d47d42 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:13.940807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.014881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d8a42-4d7b-4c2f-a9da-3ac4d3d47d42 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.032718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686b5e-33e2-4ec5-b668-2f5e2d96eed5 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.033027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686b5e-33e2-4ec5-b668-2f5e2d96eed5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.033307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686b5e-33e2-4ec5-b668-2f5e2d96eed5 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.034688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 512/1010] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:14.040491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.040821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.041109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.051573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.052517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.052985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.053419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.054307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.055866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.057826 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:14.059019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:14.088593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 499/1011] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:39:13 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:14.146786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 513/1012] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:14 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:14.170893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.274037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.286334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708361c7-a422-43b7-891e-3ad5497d82f8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.322953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.323895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.324185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.324466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] PATH_INFO: `/v3/services/compute` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.325166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.326571 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:14.339654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:14.340872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:14.341816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 500/1013] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:14.348927 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.382699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.457905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386823-db2d-44a7-a8c6-de971c4068bb None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.496303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.497503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.498002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.498506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.499880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.500215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.501670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.512184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 514/1014] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.513848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.645227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd79c9b-a95f-4629-adc0-f7db7da7dc6f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.688955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.689635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.689921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.690195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.690911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:14.691219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.692626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f47d9b-04b2-44cb-862a-2d568c35a4b8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.703185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 501/1015] 10.222.0.21 () {56 vars in 1112 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:14.704840 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:14.800787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5713-084a-4b84-a036-b9844fa628de None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.801313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5713-084a-4b84-a036-b9844fa628de None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.801313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5713-084a-4b84-a036-b9844fa628de None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.802779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 502/1016] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:14.808088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.808424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.808720 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.809538 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.817544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887567a0-04f1-476e-b86f-cfb68f3d66fe None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:14.826828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:14.828700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:14.845404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:14.856043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:14.856824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:14.857107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:14.857385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:14.858123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:14.859693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:14.888595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 515/1017] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:14.923956 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 503/1018] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:14.948094 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.059559 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d15e5-74d7-460e-9778-31bfbdc34613 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.098099 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.098905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.099194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.099471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.100664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.102716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef0506-711e-4a9a-b8dd-694dc93759f6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.104666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 516/1019] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:15.110461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:15.224447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26170014-c6b4-4071-8e2d-3baccf347b59 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:15.261462 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:15.262124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:15.262404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:15.262675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:15.263373 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:15.264839 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7b217-576a-4e66-943e-5e10c33cde7c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:15.288195 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:15.297231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 504/1020] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:39:15 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:16.396994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9162-0934-457b-9b13-401fc502ca2f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.397552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9162-0934-457b-9b13-401fc502ca2f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.397552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c9162-0934-457b-9b13-401fc502ca2f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.399752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 517/1021] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:16.404964 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.405382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.405568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.422658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:16.423855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:16.495601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 505/1022] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:16.518031 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c9162-0934-457b-9b13-401fc502ca2f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.550136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.639383 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c9162-0934-457b-9b13-401fc502ca2f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.679916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.680774 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.681050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.681330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] PATH_INFO: `/v3/services/image` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.682203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.683628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:16.696870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:16.698076 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:16.704929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 518/1023] 10.222.0.21 () {56 vars in 1105 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:16.706018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.816281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71731d26-ded5-46fc-8d01-22968f6ffdfb None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:16.847174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.858282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:16.858977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:16.859262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:16.859541 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:16.860274 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] RBAC: Adding query filter params (name=image) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:16.860617 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:16.862051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:16.873131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 506/1024] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:16.874707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:16.974551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:16.993812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeb13c-a610-489c-9554-fbf17e6f980a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.033853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.034577 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.034866 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.035142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.035983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] RBAC: Adding query filter params (type=image) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:17.036252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.037649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.047972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 519/1025] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:17.049063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.172475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f161-756e-40dc-a189-11d00f608aea None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.209248 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.209922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.210301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.210568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.211517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] RBAC: Adding query filter params (interface=public, service_id=f30a6a4bc6f74e5c96cd89b50e7dfd07, region_id=RegionOne) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:17.211833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.213267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5414d-bc04-468a-9e97-7b6c488cb0dd None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.223475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 507/1026] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/endpoints?service_id=f30a6a4bc6f74e5c96cd89b50e7dfd07&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:17.224893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.273153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b73b15-f38f-4529-b472-44e261379996 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.273428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b73b15-f38f-4529-b472-44e261379996 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.273914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b73b15-f38f-4529-b472-44e261379996 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.275675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 508/1027] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:17.280879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.281167 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.281460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.299040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:17.300234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:17.357322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14d9a-527d-40a8-b128-dd50b0c56aa9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.376594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.399201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 509/1028] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:17.404098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.407546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.407546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.407546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.407546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.407546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df876fd-fd19-4ed5-aa28-86bf9bad4ab5 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.415861 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 520/1029] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:39:17.424445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.570661 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f7209-b696-46c5-bcd4-64a44caec3c0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.583282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e27b-3063-4a61-b6e2-47560cf24437 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.583682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e27b-3063-4a61-b6e2-47560cf24437 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.583874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244e27b-3063-4a61-b6e2-47560cf24437 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.586239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 521/1030] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:17.595280 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.595529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.595858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.620335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:17.620938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:17.624204 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.625073 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.625356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.625632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.630667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.643295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:17.651946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:17.653030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:17.653909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 510/1031] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:17.666409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.712174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 522/1032] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:17.726231 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:17.736801 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.796445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffd110-e503-48e1-a710-8bb64fb152f8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.846636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.847693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.848212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.848780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.849867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:17.850181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.851636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.864900 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 511/1033] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:17.865884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.877318 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38e202-9204-4ff4-ba97-290f104484ff None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:17.925189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:17.926109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:17.926387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:17.926671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:17.929117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:17.931203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:17.933305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 523/1034] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:17.939677 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:17.957266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.007040 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338dfcc-ba33-4200-9142-cf4a7ff0eceb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.018031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.046418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.047221 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.047506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.047789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.058720 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.060409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.061972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27dab8-6141-417a-8693-872d57680847 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:18.079197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:18.080128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:18.081090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:18.081090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:18.081090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:18.081090 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:18.083049 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 512/1035] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:18.088968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.104135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.104771 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.105062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.105359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.106066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.107503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.133022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 524/1036] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:39:17 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:18.206776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ff0cb-e413-4d3d-ae52-6090251a1e45 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.253299 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.254408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.254904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.255377 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.256758 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.257294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.260237 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.274616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 513/1037] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:18.280226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.370477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.411948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb2fff-dda3-4f2d-9d70-cb079be421a9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.449484 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.450296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.450584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.450862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.459007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.460468 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:18.475034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:18.476114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:18.477141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 525/1038] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:18.484010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.508185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.601450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32353-7513-4808-a13e-bc178bbbafa7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.638044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.638643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.638931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.639210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.640055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:18.640369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.641847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:18.661635 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 514/1039] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:18.667997 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:18.801200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fcc4f-0325-4df7-a874-b0cb42b33e2e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:18.839898 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:18.854781 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:18.855621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:18.855915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:18.856223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:18.924670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:18.926151 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.073500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 526/1040] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:19.082474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.169525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.198577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7f1f0-b70f-4a51-a136-6adf21b06ead None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.233465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.234269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.234554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.234909 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.246019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.247565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:19.266876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.267822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.273351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:19.273351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:19.273351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:19.273351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:19.273351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 515/1041] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.274154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.301618 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.380510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40f68-9205-4bb6-ac99-185d28dbea50 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.420838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.421621 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.421810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.422087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.422843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.423154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.424602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.442019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 527/1042] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:19.443511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.555291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f62a0-b28a-4a2e-a0a5-8b81dde6e2cf None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.587572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.597450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.598264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.598569 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.598848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.606936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.608413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-9a4d08fc-ca72-4e3b-9ef9-1e75e3c2da9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:19.623136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:19.624088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:19.629427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 516/1043] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:19.631644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.707077 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd42ee-6ff5-4edc-b72e-d9b15b4d066e None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.707365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd42ee-6ff5-4edc-b72e-d9b15b4d066e None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.707643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd42ee-6ff5-4edc-b72e-d9b15b4d066e None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.709180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 517/1044] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:19.715030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.715312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.715598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.732253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:19.733476 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:19.734687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:19.749023 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fef1e3-f4cd-4bfa-9e5e-378e7432f9fe None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.799373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.800414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.800926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.801402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.802402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.802828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.805389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.819585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 528/1045] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:19.825743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.832630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 518/1046] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:19.851021 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:19.935479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5def88-5a45-4132-bced-364c5fe6f5ce None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.965999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae68b8d-c67f-4e9c-8625-e96bb1c2b64b None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:19.972030 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:19.972725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:19.973012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:19.973286 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:19.974121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] RBAC: Adding query filter params (role.id=ad75c144a36048c7bc05db8f7e59d70d, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:19.974436 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:19.975901 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:19.981960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 529/1047] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=ad75c144a36048c7bc05db8f7e59d70d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:20.001601 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.020080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.020936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.021228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.021513 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] PATH_INFO: `/v3/services/image` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.022190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.023662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.036518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.037613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:20.044358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 519/1048] 10.222.0.21 () {56 vars in 1105 bytes} [Wed May 25 13:39:19 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.046926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.061580 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.164309 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ccb60-a1ef-46e8-85d5-6dff8459cbe8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.203084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.203751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.204034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.204311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.205057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] RBAC: Adding query filter params (name=image) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.205362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.206739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.212143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 530/1049] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.217889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.324731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac08-fbfd-4af9-bc71-8b42e577914a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.364385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.365104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.365389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.365663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.366389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] RBAC: Adding query filter params (type=image) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:20.367427 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.369149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f1f7c-572b-439d-8e1b-2b9bd5a8058a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.374941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 520/1050] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:20.381010 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.467652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.507725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364b71c-b36f-4cfb-bbcb-6d79a96e819d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.544960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.545577 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.545868 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.546152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.546907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:20.547008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61febd96-2373-4fc6-9e8f-85665442d87b None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.547256 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61febd96-2373-4fc6-9e8f-85665442d87b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.547535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61febd96-2373-4fc6-9e8f-85665442d87b None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.548305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:20.549192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 521/1051] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:20.554227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.554535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.554816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.569879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 531/1052] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:39:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:20.575228 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:20.576375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:20.637656 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.675362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 522/1053] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:20.700508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:20.840713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e0ff55-e97b-4f3e-a48e-1c6862f973f1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:20.873281 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:20.880155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:20.880994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:20.881791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:20.883112 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:20.894050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:20.901782 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:20.902766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:20.903611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 532/1054] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:20.909054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.016320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d32f2-a37d-4b63-8135-98913c9c3f66 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.052393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.053103 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.053391 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.053669 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.054981 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.055286 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.056687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.069795 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 523/1055] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.071599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.190905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a921d95-8e31-43a0-8a87-634ea27c1f9f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.226620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.227527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.227808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.228080 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] PATH_INFO: `/v3/users/32f90977ba8d42c9875074d937d6a46f` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.242611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=32f90977ba8d42c9875074d937d6a46f)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.244074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.245995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 533/1056] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/users/32f90977ba8d42c9875074d937d6a46f => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.251470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.287713 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.358013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f8504-2eb4-417e-99eb-8ad8701561c5 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.394775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.395429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.395714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.395991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.470469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.471904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.508254 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.578851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 524/1057] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:21.584526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.640085 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.703087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af633a30-b5c8-496c-a30d-ba8116a61cf6 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.743438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.744219 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.744504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.744810 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.746046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.747477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3c1bf-8411-4dfa-9681-101f0b786da7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.749362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 534/1058] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:21.755243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:21.862967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b31295-6f5d-4854-89b9-6211b2a8eda2 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:21.898854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:21.899551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:21.899837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:21.900115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:21.900988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723, user.id=32f90977ba8d42c9875074d937d6a46f) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:21.901304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:21.902764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbffd95-aeb2-449b-818a-630d86a00a19 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:21.906063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:21.916618 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 525/1059] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:21 2022] GET /identity/v3/role_assignments?user.id=32f90977ba8d42c9875074d937d6a46f&scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:22.263892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2daa-5e83-4ed7-9ab8-b27ce842ac04 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.264226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2daa-5e83-4ed7-9ab8-b27ce842ac04 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.264464 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db2daa-5e83-4ed7-9ab8-b27ce842ac04 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.265912 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 535/1060] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:22.271133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.271428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.271708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.289377 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:22.290584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:22.360005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 526/1061] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:22.382988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2daa-5e83-4ed7-9ab8-b27ce842ac04 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.505174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db2daa-5e83-4ed7-9ab8-b27ce842ac04 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.534591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.564541 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.565944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.566432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.566915 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.581870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:22.598488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:22.599800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:22.600703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:22.604977 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 536/1062] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:22.609979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.722144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc641d-2c25-40c6-a1a5-2ad9338862f4 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.753051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:22.760800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.761551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.761863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.762171 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.762927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:22.763239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.764705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e01ffd-772c-4c5d-813b-be96ee985ef6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.774451 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 527/1063] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:22.780102 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:22.900578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b594c8-d86e-4e1f-8a37-312e06364dad None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.900794 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b594c8-d86e-4e1f-8a37-312e06364dad None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.901064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b594c8-d86e-4e1f-8a37-312e06364dad None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.902441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 528/1064] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:22.905492 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ed9b-9a3c-44bf-b7d3-a9bf5bdf4c70 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:22.907754 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.908046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.908320 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.926223 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:22.927408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:22.947305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:22.948105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:22.948389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:22.948705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:22.961163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:22.962749 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:22.981225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:22.982198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:22.983136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:22.983136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:22.983136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:22.983136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:22.983136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 537/1065] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:22.988223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.009155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 529/1066] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:23.020104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.032548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.110350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0d64-c6cb-4021-8ba1-7df8ce873f39 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.144245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b184c35-6178-4137-8ab9-06cf3350aa1d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.149140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.149817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.150100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.150373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.151150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.151407 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.153301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.167891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.172902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 538/1067] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:23.176179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.183047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.184082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.184496 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.184932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.276703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.282071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.283668 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.291213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eaef1-208e-4453-b126-ae4d20fae8aa None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.329414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.330222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.330512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.330791 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.339830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.341470 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:23.357429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:23.358416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:23.359356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 539/1068] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:23.364312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.391968 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 530/1069] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:23.397971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.423021 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.477177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76e2aa-4851-445e-8969-c009253b1467 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.503944 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380a0b0-b2f8-4950-b315-b2b4b4f6f419 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.515409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.516082 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.516366 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.516684 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.517642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.517953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.519494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.541515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.542319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.542689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.542970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.547895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 540/1070] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:23.549264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.563979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.576010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.578711 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:23.613382 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:23.614732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:23.615613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:23.615613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:23.615613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.615613 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:23.617010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 531/1071] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:23.624298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.718457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dd42b-856d-487e-8446-35d3b0040f29 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.729501 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.757522 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58c5ea-1dba-439b-93b2-27ef7a928678 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.757986 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.759122 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.759564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.760123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.775222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.777045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.794107 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.794773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.795060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.795337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.796066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.796376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:23.797352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:23.798506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:23.799459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:23.799459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:23.799459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:23.799459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:23.799689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.803430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 541/1072] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:23.804227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:23.812326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 532/1073] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:23.882856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:23.931006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35881458-ab4a-465d-9d8b-0d1b332c1f94 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:23.970311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:23.970913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:23.971203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:23.971488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:23.972235 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:23.972550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:23.973985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:23.987842 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 542/1074] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:23.993973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.114141 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a05079-f601-426c-a46f-a25770f209cc None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.152139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.153006 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.153290 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.153562 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.155558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.160965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.162849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-573ef043-fa42-4088-a36e-79321d3dd5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:24.178159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:24.180117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:24.181384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 533/1075] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:24.186544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.230339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c752-680f-47a7-92ad-678d625d7a46 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.230564 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c752-680f-47a7-92ad-678d625d7a46 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.230813 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286c752-680f-47a7-92ad-678d625d7a46 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.232658 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 534/1076] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:24.239050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.239344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.239631 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.258792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:24.260115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:24.305034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab418f15-24cf-4aa1-9735-8cee79150161 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.323243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.336833 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 535/1077] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:24.342128 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.342793 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.343079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.343376 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.344142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:24.344452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.345909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.358981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 543/1078] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:24.359711 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.364648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.461942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da79c52-a3da-4c50-97c0-864d7ac2eca1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.467263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eefef-f6e1-468c-9778-b39743d2e9cd None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.486526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.497568 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.498374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.498660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.498936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.503012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.503938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.504223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.504499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/ad75c144a36048c7bc05db8f7e59d70d` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.506839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=a21198d279bd499fbd1e2f42798f3918, role_id=ad75c144a36048c7bc05db8f7e59d70d)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.508377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.509825 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:24.518273 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:24.520188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:24.521342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 536/1079] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:24.526250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.529308 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 544/1080] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:39:24 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/a21198d279bd499fbd1e2f42798f3918/roles/ad75c144a36048c7bc05db8f7e59d70d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:24.651936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03628ff2-5ce2-4e99-86e6-0892d2ab0674 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.690676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.691367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.691653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.691931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.692749 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:24.693065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.695143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.710331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 537/1081] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:24.711983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.758280 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.823930 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac9967-f9d5-47b0-9940-b04e79e38432 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:24.861072 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:24.861956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:24.862237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:24.862514 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] PATH_INFO: `/v3/users/32f90977ba8d42c9875074d937d6a46f` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:24.863710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] RBAC: Authorizing `identity:get_user(user_id=32f90977ba8d42c9875074d937d6a46f)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:24.865205 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:24.867555 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 545/1082] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/users/32f90977ba8d42c9875074d937d6a46f => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:24.874115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:24.925918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:24.986057 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dca50-ad0e-4536-9dd4-d61797f91a6c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.025168 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.025953 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.026235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.026572 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.028053 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.029560 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a707b8f-846b-4b77-a663-7a617c662a04 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.031895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 538/1083] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:24 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:25.038820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:25.150728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4f905-a6eb-4cc6-b25f-27623ac63c8a None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:25.158719 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:25.187422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:25.188307 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:25.188606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:25.188885 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723/users/32f90977ba8d42c9875074d937d6a46f/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:25.190869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f0f76e6bb4402bcad1ce78ca97723, user_id=32f90977ba8d42c9875074d937d6a46f, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:25.192304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322dc94-446d-4aab-88fd-b8d04391cbbd None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:25.207970 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 546/1084] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:39:25 2022] PUT /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723/users/32f90977ba8d42c9875074d937d6a46f/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:26.786203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a111-1947-4585-93dd-a6ef8c73d515 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.786727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a111-1947-4585-93dd-a6ef8c73d515 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.786871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186a111-1947-4585-93dd-a6ef8c73d515 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.788338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 539/1085] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:26.793668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:26.794098 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:26.794351 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:26.811338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:26.812533 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:26.878507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 547/1086] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:26.902413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186a111-1947-4585-93dd-a6ef8c73d515 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.005901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186a111-1947-4585-93dd-a6ef8c73d515 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.020302 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.043764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.045196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.045667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.045961 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.058518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:27.066712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.067870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:27.069097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 540/1087] 10.222.0.21 () {56 vars in 1120 bytes} [Wed May 25 13:39:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.074177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.167553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.174960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb5fb-fb51-4869-ac99-ad046d18a110 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.210136 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.210830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.211121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.211403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.212187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.212506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.213973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.223632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 548/1088] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.230139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.337190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd5135-61b8-4a55-977d-d6cf5c84d99b None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.375294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.376080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.376359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.376662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.387270 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.388753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-7ab743f2-11d3-4f18-ab40-d8d72ebc6bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:27.408341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.409544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.410570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:27.410570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:27.410570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:27.410570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:27.410570 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 541/1089] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.416850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.429888 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b340-2098-4463-966d-ce2ff3f2f968 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.430074 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b340-2098-4463-966d-ce2ff3f2f968 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.430355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646b340-2098-4463-966d-ce2ff3f2f968 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.431712 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 542/1090] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:27.436990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.437278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.437551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.454810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:27.456005 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:27.477895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.517597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d518e-e991-4db9-b79b-842e562d7f50 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.525807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 543/1091] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:27.550418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.552796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.553566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.553938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.554214 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.554935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.555239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.556646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.568471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 549/1092] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:27.574127 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.593221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.654507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8cd9f6-97e3-45cd-ac23-51108ec4302c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.674602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b859aa7-b821-4d69-bdbf-84d410c9c6e8 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.688474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.689214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.689495 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.689779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.701193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:27.710851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.711902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:27.712894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 544/1093] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:27.715088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.715088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.715088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.715088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] PATH_INFO: `/v3/users/nova` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.718587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.726017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.727499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.738986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:39:27.743206 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:27.744359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:27.745569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 550/1094] 10.222.0.21 () {56 vars in 1093 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:27.750118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.824827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c87dc3-67ab-4e39-bab3-06555705eaee None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.829753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:27.851926 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c98af2-f600-4461-aa9f-02843e06b29f None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:27.860416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.861120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.861421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.861698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.862450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.862764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.864162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.873413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 545/1095] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:27.879325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.888989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:27.890044 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:27.890523 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:27.890982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:27.892413 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:27.892972 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:27.895444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:27.923839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 551/1096] 10.222.0.21 () {56 vars in 1133 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:27.931090 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:27.986587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.009716 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1862931-7feb-4bec-85f1-8cbd7b75c586 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.041849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca51de3-5951-4674-af17-f861947c4cb0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.043510 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.044384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.044699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.044976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] PATH_INFO: `/v3/users/32f90977ba8d42c9875074d937d6a46f` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.046203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] RBAC: Authorizing `identity:get_user(user_id=32f90977ba8d42c9875074d937d6a46f)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.047706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.049424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 546/1097] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/users/32f90977ba8d42c9875074d937d6a46f => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:28.055217 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.081607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.082453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.082736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.083017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.087216 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.151907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.153445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.156389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5604b33-f6cc-46d9-944f-ce7318e27a9e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.191052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.192095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.192385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.192795 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.208736 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.253115 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 552/1098] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:39:28.258763 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.261054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.262606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.292660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.356916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a73e53-7283-45d4-96aa-37c689140fd0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.362549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 547/1099] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:39:28.368403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.391924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.393153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.393541 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.393908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.405794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.407277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.415123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:39:28.425036 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.426091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.431854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:39:28.431854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:39:28.431854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:39:28.431854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:28.431854 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 553/1100] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:28.432198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.469487 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fbdc8-953e-4989-9fe7-30779a2bd150 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.503016 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.503929 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.504166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.504446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.505670 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.507166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.508923 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 548/1101] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:28.514816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.523495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.535661 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8f289-e30e-4def-8a56-e5d8cc017c9b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.572548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.573241 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.573530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.573812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.574532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.574843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.576232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.588590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 554/1102] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.594110 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.617280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc8f48-749d-42d4-a292-4628a0c3a9f4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.631343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:28.651755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.652409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.652723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.653004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.653830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723, user.id=32f90977ba8d42c9875074d937d6a46f) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.654141 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.655565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.667799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 549/1103] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/role_assignments?user.id=32f90977ba8d42c9875074d937d6a46f&scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:28.703251 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6799f-ff35-4923-9aab-a1df304aaf02 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.740167 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.741748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.742041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.742316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.752639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.754191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:39:28.775028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:28.776118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:28.777283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 555/1104] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:28.782339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.888389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e1504-89f0-465a-9107-5388440eac46 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:28.922762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:28.923429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:28.923713 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:28.923991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:28.924805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:28.925121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:28.926573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f57b1-1ff2-46ea-96b4-49497349e423 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:28.939392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 550/1105] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:28.945718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:28.956761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:29.046220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99687f61-0615-4ab5-8971-a890f2414cb7 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:29.081933 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:29.082592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:29.082875 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:29.083150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:29.083979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] RBAC: Adding query filter params (role.id=ad75c144a36048c7bc05db8f7e59d70d, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=a21198d279bd499fbd1e2f42798f3918) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:29.084827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:29.086731 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d5474-2776-4733-b4d2-402843cda6d4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:29.093162 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 556/1106] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:28 2022] GET /identity/v3/role_assignments?user.id=a21198d279bd499fbd1e2f42798f3918&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=ad75c144a36048c7bc05db8f7e59d70d => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:30.957566 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8a2c-b3b9-441f-90af-c0c97c2c9e00 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.958107 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8a2c-b3b9-441f-90af-c0c97c2c9e00 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.958107 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b8a2c-b3b9-441f-90af-c0c97c2c9e00 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.960343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 551/1107] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:30.965833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:30.966262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:30.966402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:30.985333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:30.986757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:31.059596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 557/1108] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:31.089388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b8a2c-b3b9-441f-90af-c0c97c2c9e00 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.135179 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.199957 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b8a2c-b3b9-441f-90af-c0c97c2c9e00 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.235472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.236203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.236486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.236802 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.238061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.239536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a30d31-2931-4fa0-852d-38440dd2f61d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.241770 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 552/1109] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:31.247691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:31.249911 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:31.354211 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da0eb6-1bc9-453e-9a44-2d3b1293dc10 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:31.390548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:31.391234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:31.391520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:31.391799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:31.392530 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:31.394089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869fc5-1940-4fe6-adb9-d0ac7dabd0f4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:31.415784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 558/1110] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:39:31 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:33.668172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb80d-f7c3-4227-aa5f-08096c0e872d None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.668728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb80d-f7c3-4227-aa5f-08096c0e872d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.668847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb80d-f7c3-4227-aa5f-08096c0e872d None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.670136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 553/1111] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:33.675302 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.675728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.675856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.693256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:33.694709 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:33.748418 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.762843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 559/1112] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:33.786106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bb80d-f7c3-4227-aa5f-08096c0e872d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:33.894136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bb80d-f7c3-4227-aa5f-08096c0e872d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:33.902393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:33.929461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:33.930315 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:33.930600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:33.930880 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] PATH_INFO: `/v3/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:33.932362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] RBAC: Authorizing `identity:get_role(role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:33.933774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:33.935149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 554/1113] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/roles/13f5475a26344e9e816475b8539030d3 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:33.940984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.042646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7d6e3-09ba-4090-baaf-66649676c86c None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.078344 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.079232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.079522 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.079802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] PATH_INFO: `/v3/users/3c5a20d596b241d5aa998f97dcbde7bc` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.093317 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] RBAC: Authorizing `identity:get_user(user_id=3c5a20d596b241d5aa998f97dcbde7bc)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.094787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.096507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 560/1114] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:33 2022] GET /identity/v3/users/3c5a20d596b241d5aa998f97dcbde7bc => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:34.102473 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.199856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb0e99-e4ad-4c56-93b1-425414cb6a53 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.232867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.233504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.233791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.234072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.297272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.351355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.354218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.522442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 555/1115] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:34.528584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.573980 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.627550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f4929-397a-4142-bdb4-6024a497313d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.664362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.665256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.665539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.665816 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.666981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.668409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b50773-7e12-4953-a587-41ab454a2a31 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.670149 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 561/1116] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:34.676277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:34.752463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:34.781019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9468cdc-7ed8-440a-9346-b2369653bf9f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:34.821239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:34.821913 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:34.822202 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:34.822488 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:34.823344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723, user.id=3c5a20d596b241d5aa998f97dcbde7bc) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:34.823663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:34.825215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac5188-99bf-4bf3-875e-59de1f3c177d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:34.832093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 556/1117] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:34 2022] GET /identity/v3/role_assignments?user.id=3c5a20d596b241d5aa998f97dcbde7bc&scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723&role.id=13f5475a26344e9e816475b8539030d3 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.014693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edc38-21b9-49d0-b774-da21ab2a6b9e None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.015259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edc38-21b9-49d0-b774-da21ab2a6b9e None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.015259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edc38-21b9-49d0-b774-da21ab2a6b9e None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.016641 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 562/1118] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:37.022334 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.022788 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.022898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.040045 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:37.042126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:37.115156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 557/1119] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:37.141568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7edc38-21b9-49d0-b774-da21ab2a6b9e None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.242958 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7edc38-21b9-49d0-b774-da21ab2a6b9e None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.257996 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.280377 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.281203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.281492 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.281776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] PATH_INFO: `/v3/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.283348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] RBAC: Authorizing `identity:get_role(role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.284785 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.286230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 563/1120] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/roles/13f5475a26344e9e816475b8539030d3 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:37.292428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.361593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.396988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faea8e2-6b96-4294-a620-27426e1ed663 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.448362 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.449261 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.449541 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.449815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] PATH_INFO: `/v3/users/3c5a20d596b241d5aa998f97dcbde7bc` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.451255 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] RBAC: Authorizing `identity:get_user(user_id=3c5a20d596b241d5aa998f97dcbde7bc)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.452739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.454371 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 558/1121] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/users/3c5a20d596b241d5aa998f97dcbde7bc => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.460336 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.565628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420b57-d1d7-4a66-9d2c-f72d6fb201e9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.599907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.600797 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.601084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.601360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.602537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.603957 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1c425-8fb4-4620-9b25-b94c78556ccc None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.605679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 564/1122] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:37 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:37.611523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:37.711755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b282b1-9fe4-45ed-9741-9a6ef233c65a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:37.746419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:37.747309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:37.747591 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:37.747866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723/users/3c5a20d596b241d5aa998f97dcbde7bc/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:37.750055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f0f76e6bb4402bcad1ce78ca97723, user_id=3c5a20d596b241d5aa998f97dcbde7bc, role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:37.751499 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7c412-9973-4ac3-b6d8-f65f22131e83 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:37.754629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:37.765751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 559/1123] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:39:37 2022] PUT /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723/users/3c5a20d596b241d5aa998f97dcbde7bc/roles/13f5475a26344e9e816475b8539030d3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:39.984586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89741-1231-440c-96a7-fbb0ba6113db None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.985138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89741-1231-440c-96a7-fbb0ba6113db None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.985138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f89741-1231-440c-96a7-fbb0ba6113db None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:39.986553 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 565/1124] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:39.993834 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:39.994445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:39.994445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.011651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:40.012906 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:40.085759 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 560/1125] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:40.111823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f89741-1231-440c-96a7-fbb0ba6113db None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.174273 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.229501 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f89741-1231-440c-96a7-fbb0ba6113db None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.276163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.276952 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.277234 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.277512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] PATH_INFO: `/v3/roles/13f5475a26344e9e816475b8539030d3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.279011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] RBAC: Authorizing `identity:get_role(role_id=13f5475a26344e9e816475b8539030d3)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.280391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.281805 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 566/1126] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/roles/13f5475a26344e9e816475b8539030d3 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:40.288178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.392825 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7f29e-4b54-44c8-90c5-f211775a0318 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.420380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.430801 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.431681 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.431965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.432247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] PATH_INFO: `/v3/users/3c5a20d596b241d5aa998f97dcbde7bc` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.433542 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=3c5a20d596b241d5aa998f97dcbde7bc)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.434950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.436616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 561/1127] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/users/3c5a20d596b241d5aa998f97dcbde7bc => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:40.442704 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.564892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32186e-7aa2-447d-8eec-0d70c75f7497 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.587790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:40.610155 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.610838 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.611123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.611403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.689143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.690688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.801878 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 567/1128] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:40.808503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.917685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c55961-4e5e-40fc-a8d1-c921e7754ea8 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:40.953288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:40.954159 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:40.954447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:40.954727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] PATH_INFO: `/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:40.955940 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] RBAC: Authorizing `identity:get_project(project_id=fd3f0f76e6bb4402bcad1ce78ca97723)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:40.957447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f553b2-ee43-45c8-afdf-e46383cd5d3e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:40.959783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 562/1129] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/projects/fd3f0f76e6bb4402bcad1ce78ca97723 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:40.966225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:40.986139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:41.074284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cf2a0-fa76-4c6a-9986-6fec7feda067 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:41.111784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:41.112453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:41.112778 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:41.113062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:41.113914 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] RBAC: Adding query filter params (role.id=13f5475a26344e9e816475b8539030d3, scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723, user.id=3c5a20d596b241d5aa998f97dcbde7bc) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:41.114295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:41.115939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe1ed6-ec29-47bc-9908-2a627630f4d4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:41.122671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 568/1130] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:40 2022] GET /identity/v3/role_assignments?user.id=3c5a20d596b241d5aa998f97dcbde7bc&scope.project.id=fd3f0f76e6bb4402bcad1ce78ca97723&role.id=13f5475a26344e9e816475b8539030d3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:43.451919 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43366a9-6214-4005-a69a-e769b174ee6f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.452464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43366a9-6214-4005-a69a-e769b174ee6f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.452588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43366a9-6214-4005-a69a-e769b174ee6f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.454042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 563/1131] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:39:43.460693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.461258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.461258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.479361 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:43.480623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:43.551180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 569/1132] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:43.576805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43366a9-6214-4005-a69a-e769b174ee6f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.647349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.694959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43366a9-6214-4005-a69a-e769b174ee6f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.734366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.735089 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.735378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.735659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.736937 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.738410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72b4b8-74e6-4fcd-bc08-e8cbd8cafa2e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.740100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 564/1133] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:43.746109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:43.761981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:43.858493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df9253-4880-4a0e-9f0f-2c5083159119 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:43.894330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:43.895047 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:43.895333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:43.895616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:43.896460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:43.897955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a9ad-6365-4814-90b8-a48720afba9a None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:43.925129 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 570/1134] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:39:43 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:39:46.544120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203c2f8-7ed0-4276-9889-65ecd25cb8b8 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.544753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203c2f8-7ed0-4276-9889-65ecd25cb8b8 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.544753 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203c2f8-7ed0-4276-9889-65ecd25cb8b8 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.546060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 565/1135] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:46.551382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.551824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.551925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.568825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:46.570017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:46.630686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.639924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 571/1136] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:39:46.665349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203c2f8-7ed0-4276-9889-65ecd25cb8b8 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.771699 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203c2f8-7ed0-4276-9889-65ecd25cb8b8 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.799706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:46.807529 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.808344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.808659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.808936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.810206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.811690 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fc1fe-a471-4942-9bb8-6607e39aa412 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.813430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 566/1137] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:46.820339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:46.920035 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec9fea-7275-4942-af40-6aaaf09facd5 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:46.954046 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:46.954716 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:46.955000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:46.955284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:46.956000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:46.958007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30363166-a493-4b7e-affd-bf68d2c94e65 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:46.978823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 572/1138] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:39:46 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:39:49.255443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3701f5-3497-411b-b210-78ec9bd13bbe None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.255988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3701f5-3497-411b-b210-78ec9bd13bbe None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.255988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3701f5-3497-411b-b210-78ec9bd13bbe None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.257408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 567/1139] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:49.262788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.263220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.263347 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.279592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:49.280794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:49.325666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.347123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 573/1140] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:49.370675 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3701f5-3497-411b-b210-78ec9bd13bbe None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.469190 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3701f5-3497-411b-b210-78ec9bd13bbe None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.502285 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.503060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.503341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.503616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.514102 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:49.522230 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:49.523532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:49.524700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 568/1141] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:49.529602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.633362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c504273d-e029-4d91-89bf-b12e6befc3cb None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.667045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.667710 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.667997 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.668278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.669059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:49.669373 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:49.670746 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:49.679458 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 574/1142] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:49.685288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.747376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.789126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc06d50-4aa6-4ffa-b21d-5941c622139c None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.820905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.821756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.822035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.822307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] PATH_INFO: `/v3/users/5b7565b12659408991c1d353a3c07292` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:49.835939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] RBAC: Authorizing `identity:get_user(user_id=5b7565b12659408991c1d353a3c07292)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:49.837408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:49.839036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 569/1143] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/users/5b7565b12659408991c1d353a3c07292 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:49.844799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:49.887537 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:49.943408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac990ef-19ec-49cd-93bb-89fbc005c968 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:49.976902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:49.977558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:49.977845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:49.978124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.043066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.044463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.085059 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.141723 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 575/1144] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:50.147877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.247323 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b5693-0287-459a-9fc7-10c00a1e9195 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.263365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:50.281845 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.282693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.282977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.283251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] PATH_INFO: `/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.284422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] RBAC: Authorizing `identity:get_project(project_id=d66982dbbed14d5ab9bd1fe48a8fba96)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.286596 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443b158-75f2-47fb-9815-d76360d5616e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.288313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 570/1145] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:50.294532 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:50.391686 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ae78-6722-49da-9e94-fd0efa085597 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:50.428298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:50.428983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:50.429274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:50.429556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:50.430382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=d66982dbbed14d5ab9bd1fe48a8fba96, user.id=5b7565b12659408991c1d353a3c07292) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:50.430697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:50.432135 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c8142-61e8-4f64-a9da-f8056d5a1675 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:50.437548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 576/1146] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:50 2022] GET /identity/v3/role_assignments?user.id=5b7565b12659408991c1d353a3c07292&scope.project.id=d66982dbbed14d5ab9bd1fe48a8fba96&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:52.697453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f632-2a26-4334-9890-e30fe39d995d None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.698066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f632-2a26-4334-9890-e30fe39d995d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.698066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5942f632-2a26-4334-9890-e30fe39d995d None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.699441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 571/1147] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:52.705817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.706656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.706656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.719570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:52.725448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:52.726682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:52.799008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 577/1148] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:52.822526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f632-2a26-4334-9890-e30fe39d995d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:52.930186 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5942f632-2a26-4334-9890-e30fe39d995d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:52.966614 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:52.967410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:52.967696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:52.967974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:52.979295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:52.987001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:52.988878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:39:52.990288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 572/1149] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:39:52.997690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.124489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c60e64-e095-4fef-a91c-d33cebd5e278 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.161504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.162183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.162474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.162756 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.163509 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:53.163824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.165246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.167751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:53.174960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 578/1150] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:53.181475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.229809 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:53.287863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa95b36-cfcd-43cd-a0ac-28a14dd2d3e4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.326756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.327736 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.328067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.328412 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] PATH_INFO: `/v3/users/5b7565b12659408991c1d353a3c07292` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.329777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] RBAC: Authorizing `identity:get_user(user_id=5b7565b12659408991c1d353a3c07292)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.331403 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.333883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 573/1151] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/users/5b7565b12659408991c1d353a3c07292 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:53.339864 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.458200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b7943-cabc-4b95-ba8f-09cc6c71f205 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.494026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.494963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.495294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.495622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] PATH_INFO: `/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.496900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] RBAC: Authorizing `identity:get_project(project_id=d66982dbbed14d5ab9bd1fe48a8fba96)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.498400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bab63f-ef93-4641-abbf-c7bac83a8b09 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.500165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 579/1152] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:53 2022] GET /identity/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:53.506819 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:53.615999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a06844-cf0d-4632-a289-2671f40c0c06 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:53.630366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:53.652433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:53.653361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:53.653649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:53.653929 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] PATH_INFO: `/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96/users/5b7565b12659408991c1d353a3c07292/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:53.655988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] RBAC: Authorizing `identity:create_grant(project_id=d66982dbbed14d5ab9bd1fe48a8fba96, user_id=5b7565b12659408991c1d353a3c07292, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:53.657477 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775edfe0-cc5f-4f2b-a9cf-4cee7a49933c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:53.672732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 574/1153] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:39:53 2022] PUT /identity/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96/users/5b7565b12659408991c1d353a3c07292/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:39:55.990886 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb595fb-edd8-46ff-80a7-e5a1bfcb6e9c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.991472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb595fb-edd8-46ff-80a7-e5a1bfcb6e9c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.991472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb595fb-edd8-46ff-80a7-e5a1bfcb6e9c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:55.992995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 580/1154] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:39:55.999251 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:55.999678 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:55.999810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.017332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:39:56.018580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:39:56.086892 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 575/1155] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:39:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:39:56.110705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb595fb-edd8-46ff-80a7-e5a1bfcb6e9c None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.203395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.247945 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb595fb-edd8-46ff-80a7-e5a1bfcb6e9c None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.288873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.289718 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.290003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.290277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.303165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:39:56.311109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:39:56.312882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:39:56.313752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 581/1156] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:39:56.319700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.427797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b57ce-d89c-4c8c-b87f-3fec7fc4ff8c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.429474 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.467027 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.467765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.468104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.468439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.469325 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:56.469686 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.471446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.482393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 576/1157] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:56.489226 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.603360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b109d9-ce2c-4b1b-bd70-2f5910fe9fd1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.631240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.640531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.641447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.641730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.642002 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] PATH_INFO: `/v3/users/5b7565b12659408991c1d353a3c07292` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.643232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] RBAC: Authorizing `identity:get_user(user_id=5b7565b12659408991c1d353a3c07292)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.645266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:56.646955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 582/1158] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/users/5b7565b12659408991c1d353a3c07292 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:39:56.652997 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:56.764435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a935ae7-43b1-452b-b21a-45532f1dc24f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:56.805515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:56.806245 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:56.806580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:56.806920 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:56.845272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:56.889508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:56.890931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.004653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 577/1159] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:39:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:39:57.010853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.120819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d76d4c-84fa-4111-b9cd-bf4a8414d482 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.157016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.157895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.158178 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.158453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] PATH_INFO: `/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.160359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] RBAC: Authorizing `identity:get_project(project_id=d66982dbbed14d5ab9bd1fe48a8fba96)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.161869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24699eb3-f89e-4401-9799-e1d4687fe062 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.163658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 583/1160] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/projects/d66982dbbed14d5ab9bd1fe48a8fba96 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:39:57.169996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:39:57.239471 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:39:57.281246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728328d0-0621-491e-88ae-b7b2eae520e7 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:39:57.323708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:39:57.325967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:39:57.326252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:39:57.326536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:39:57.327394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=d66982dbbed14d5ab9bd1fe48a8fba96, user.id=5b7565b12659408991c1d353a3c07292) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:39:57.327711 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:39:57.329233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718b50f-075a-4ab0-b063-d95454954d60 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:39:57.336761 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 578/1161] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:39:57 2022] GET /identity/v3/role_assignments?user.id=5b7565b12659408991c1d353a3c07292&scope.project.id=d66982dbbed14d5ab9bd1fe48a8fba96&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:00.163794 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f480f-7d10-40bd-80a7-9eeff4fdc35e None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.164403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f480f-7d10-40bd-80a7-9eeff4fdc35e None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.164403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f480f-7d10-40bd-80a7-9eeff4fdc35e None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.165777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 584/1162] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:00.171682 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.172297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.172297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.193011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:00.194472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:00.274100 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 579/1163] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:00.296895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f480f-7d10-40bd-80a7-9eeff4fdc35e None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.366121 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.407283 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f480f-7d10-40bd-80a7-9eeff4fdc35e None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.444876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.445556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.445885 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.446208 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.520396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.521994 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.598652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.641649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 585/1164] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 13:40:00.648963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.714531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:00.759530 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d2617-59ab-4158-a11f-b5a47ad90a7d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:00.795031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:00.795893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:00.796250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:00.796664 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:00.805384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:00.806970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-28723a36-9fb7-4e0a-a8a1-1b636d0308a7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:00.822727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:00.825014 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:40:00.826996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 580/1165] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:00.832845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:00.965326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d1a04-e129-467d-8867-9bb11df5fd93 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:01.031837 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:01.032989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:01.033501 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:01.034579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:01.035975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:01.036492 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:01.039244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad2041-a1bc-420c-bb88-4d96c66c66f4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:01.054457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 586/1166] 10.222.0.21 () {56 vars in 1115 bytes} [Wed May 25 13:40:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:03.399822 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b68fc-8355-4f91-a8ca-6eceaf2621d2 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.400440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b68fc-8355-4f91-a8ca-6eceaf2621d2 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.400737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b68fc-8355-4f91-a8ca-6eceaf2621d2 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.402871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 581/1167] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:03.410394 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719ee9-cf74-443b-aa97-0c8ce5aad79a None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.411156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719ee9-cf74-443b-aa97-0c8ce5aad79a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.411156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1719ee9-cf74-443b-aa97-0c8ce5aad79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.431648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a1719ee9-cf74-443b-aa97-0c8ce5aad79a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:03.432918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a1719ee9-cf74-443b-aa97-0c8ce5aad79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:03.460014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:03.522246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 587/1168] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:03.547515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b68fc-8355-4f91-a8ca-6eceaf2621d2 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:03.666881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b68fc-8355-4f91-a8ca-6eceaf2621d2 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:03.687855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:03.707567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:03.708313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:03.708676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:03.709007 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:03.709807 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:03.711271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222a62e-d105-4030-a07a-6f5b84f0edb7 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:03.728742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 582/1169] 10.222.0.21 () {60 vars in 1138 bytes} [Wed May 25 13:40:03 2022] POST /identity/v3/domains => generated 267 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:40:06.443402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0002c8-11d9-4fbf-9215-b7acb793ee79 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.444027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0002c8-11d9-4fbf-9215-b7acb793ee79 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.444179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0002c8-11d9-4fbf-9215-b7acb793ee79 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.446311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 588/1170] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:06.454213 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.454946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.454946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.472946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:06.474424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:06.552994 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 583/1171] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:06.583741 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0002c8-11d9-4fbf-9215-b7acb793ee79 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.746295 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0002c8-11d9-4fbf-9215-b7acb793ee79 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.787579 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.788617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.789003 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.789367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] PATH_INFO: `/v3/domains/c24172a47ad44627a8730fe0b2c29b83` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.802501 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c24172a47ad44627a8730fe0b2c29b83)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.804124 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745067b1-116a-4c91-bc62-916ffc512df4 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:06.806053 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 589/1172] 10.222.0.21 () {56 vars in 1183 bytes} [Wed May 25 13:40:06 2022] GET /identity/v3/domains/c24172a47ad44627a8730fe0b2c29b83 => generated 267 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:06.815934 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:06.946638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95739de-880a-4c76-a0af-0ce24e9eda7f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:06.958714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:06.989326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:06.990060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:06.990384 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:06.990704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:06.991466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:06.993066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002664b3-34c4-40be-b680-c06ebc1495a6 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:07.029742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 584/1173] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:40:06 2022] POST /identity/v3/projects => generated 361 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:09.447293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae667e-d864-4614-8759-26445e62c785 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.447861 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae667e-d864-4614-8759-26445e62c785 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.448524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae667e-d864-4614-8759-26445e62c785 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.451169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 590/1174] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:09.457660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.458452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.458452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.476672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:09.478151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:09.555220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 585/1175] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:09.581437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae667e-d864-4614-8759-26445e62c785 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:09.603014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:09.696777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae667e-d864-4614-8759-26445e62c785 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:09.735608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:09.737064 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.737351 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.737628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] PATH_INFO: `/v3/domains/c24172a47ad44627a8730fe0b2c29b83` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.739051 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c24172a47ad44627a8730fe0b2c29b83)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:09.740517 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc7a0-761e-4f4f-85aa-16787654d4f9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:09.742266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 591/1176] 10.222.0.21 () {56 vars in 1183 bytes} [Wed May 25 13:40:09 2022] GET /identity/v3/domains/c24172a47ad44627a8730fe0b2c29b83 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:09.748346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:09.869304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:09.888779 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf22de-4a5c-45d0-8de1-65980e132737 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:09.946506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:09.947244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:09.947574 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:09.947901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:09.948960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:09.950528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e082b-e7c7-4dfc-8b1f-9040e06f0725 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:09.975599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 586/1177] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:40:09 2022] POST /identity/v3/users => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:40:12.359697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c0c3-d120-4078-96dd-bcacc707bdad None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.360272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c0c3-d120-4078-96dd-bcacc707bdad None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.360272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059c0c3-d120-4078-96dd-bcacc707bdad None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.362359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 592/1178] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:12.368820 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.369663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.369663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.388069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:12.389324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:12.465840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 587/1179] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:12.491412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059c0c3-d120-4078-96dd-bcacc707bdad None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:12.590770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:12.610754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059c0c3-d120-4078-96dd-bcacc707bdad None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:12.649170 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:12.649965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.650248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.650521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.663231 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:12.671018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:12.672852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:40:12.674591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 593/1180] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:12.681969 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:12.764812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:12.792084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185f572-9b54-4d3a-949b-0b775ce30e07 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:12.829435 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:12.830422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.830708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.830990 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:12.831757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:12.832070 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:12.833534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:12.842751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 588/1181] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:12.849591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:12.958799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f238-2fdb-4336-8e4c-810d1228d366 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:12.994401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:12.995278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:12.995564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:12.995839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] PATH_INFO: `/v3/users/f9f6ee70b4ad45b0a49767a225f44571` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.010777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6ee70b4ad45b0a49767a225f44571)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.012268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.014080 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 594/1182] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:40:12 2022] GET /identity/v3/users/f9f6ee70b4ad45b0a49767a225f44571 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:13.020606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.129882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca92117-e0b0-4d83-a6f7-52e2d8dd6437 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.163332 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.163984 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.164269 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.164542 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.174172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.240605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.242120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.350577 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 589/1183] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:13.357417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.361940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.533754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57fb66-82a3-4ab8-96cc-aebc2c5821b2 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.581713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.582605 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.582895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.583175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] PATH_INFO: `/v3/projects/db7b3552e97b48da836c2970506c75aa` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.584438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] RBAC: Authorizing `identity:get_project(project_id=db7b3552e97b48da836c2970506c75aa)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.585948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cea82-962b-4223-b54d-7786975377ea None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.587726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 595/1184] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/projects/db7b3552e97b48da836c2970506c75aa => generated 361 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:13.593945 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:13.641891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:13.705863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19573809-bdf9-4387-b18f-70e875ea2404 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:13.743696 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:13.744358 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:13.744673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:13.744955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:13.745798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=db7b3552e97b48da836c2970506c75aa, user.id=f9f6ee70b4ad45b0a49767a225f44571) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:13.746111 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:13.747560 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b35843-5d36-4e00-8fee-a06113094b09 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:13.753856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 590/1185] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:40:13 2022] GET /identity/v3/role_assignments?user.id=f9f6ee70b4ad45b0a49767a225f44571&scope.project.id=db7b3552e97b48da836c2970506c75aa&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:16.373630 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366244e-0fb2-4a52-9fb1-3e90e8e5f121 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.374247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366244e-0fb2-4a52-9fb1-3e90e8e5f121 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.374247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366244e-0fb2-4a52-9fb1-3e90e8e5f121 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.375564 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 596/1186] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:40:16.382408 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.382972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.382972 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.402341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:16.403654 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:16.487246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 591/1187] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:40:16.527392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366244e-0fb2-4a52-9fb1-3e90e8e5f121 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.627461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.665415 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366244e-0fb2-4a52-9fb1-3e90e8e5f121 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.706154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.706978 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.707266 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.707546 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.720759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:16.728839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:16.730752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:40:16.732588 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 597/1188] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:40:16.736687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:16.771947 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:16.854016 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0569323-1a10-4fdc-8115-61d93c7ec7a4 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:16.892243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:16.892938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:16.893278 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:16.893612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:16.894428 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:16.894790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:16.896254 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:16.905380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 592/1189] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:16.911807 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:17.019955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57b67-30aa-4e00-bc19-d0ee24d25e82 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:17.051563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:17.058925 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:17.059827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:17.060118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:17.060401 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] PATH_INFO: `/v3/users/f9f6ee70b4ad45b0a49767a225f44571` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:17.061692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6ee70b4ad45b0a49767a225f44571)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:17.063738 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.065547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 598/1190] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:40:16 2022] GET /identity/v3/users/f9f6ee70b4ad45b0a49767a225f44571 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:17.072102 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:17.176311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:17.185424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236f4c3-9893-4f95-adc5-91e35d7c1ad3 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:17.222579 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:17.223497 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:17.223827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:17.224149 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] PATH_INFO: `/v3/projects/db7b3552e97b48da836c2970506c75aa` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:17.225482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] RBAC: Authorizing `identity:get_project(project_id=db7b3552e97b48da836c2970506c75aa)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:17.227019 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508c463-05be-4717-a0a3-f6e06f3c35ad None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.228918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 593/1191] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:40:17 2022] GET /identity/v3/projects/db7b3552e97b48da836c2970506c75aa => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:40:17.236606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:17.398604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecea37-f3c6-4ca1-b5c0-75536b5ea1c9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:17.435237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:17.436150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:17.436477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:17.436836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] PATH_INFO: `/v3/projects/db7b3552e97b48da836c2970506c75aa/users/f9f6ee70b4ad45b0a49767a225f44571/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:17.439012 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7b3552e97b48da836c2970506c75aa, user_id=f9f6ee70b4ad45b0a49767a225f44571, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:17.440504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f804ac-f673-4e64-ba25-79b9d924e782 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:17.455484 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 599/1192] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:40:17 2022] PUT /identity/v3/projects/db7b3552e97b48da836c2970506c75aa/users/f9f6ee70b4ad45b0a49767a225f44571/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:40:19.722852 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aef29-910a-43ee-85c1-3929bff0fc6c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.723466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aef29-910a-43ee-85c1-3929bff0fc6c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.723466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aef29-910a-43ee-85c1-3929bff0fc6c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.724866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 594/1193] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:40:19.730984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:19.731568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:19.731568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:19.750270 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:40:19.751511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:40:19.816274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:19.833698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 600/1194] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:40:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:40:19.857962 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76aef29-910a-43ee-85c1-3929bff0fc6c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.001884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76aef29-910a-43ee-85c1-3929bff0fc6c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.039044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.039840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.040124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.040397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.052430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:40:20.060207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:40:20.061912 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:40:20.064131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 595/1195] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:40:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:40:20.069950 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.175562 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b8403-514f-47eb-99e4-76d4f8db5d51 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.212414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.213118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.213399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.213673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.214429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:20.214737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.216131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.226088 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 601/1196] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:20.232086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.258123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.342128 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a08a67-3c80-4674-8229-08fe5a111fcd None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.378719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.379616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.379899 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.380177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] PATH_INFO: `/v3/users/f9f6ee70b4ad45b0a49767a225f44571` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.381469 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6ee70b4ad45b0a49767a225f44571)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.382959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.384711 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 596/1197] 10.222.0.21 () {56 vars in 1177 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/users/f9f6ee70b4ad45b0a49767a225f44571 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:20.390848 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.448038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.503448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566d1b0-bbec-41bc-a9cc-458104a91567 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.539404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.540057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.540342 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.540649 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.615605 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.617194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.674148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.731201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 602/1198] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:40:20.739379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:20.866750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:20.878353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069b3ba-3bf8-49a4-a607-485fc1ddf3eb None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:20.916211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:20.917144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:20.917439 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:20.917719 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] PATH_INFO: `/v3/projects/db7b3552e97b48da836c2970506c75aa` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:20.919062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] RBAC: Authorizing `identity:get_project(project_id=db7b3552e97b48da836c2970506c75aa)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:20.920598 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c3ae2-c1d5-4515-9d56-8fc3b080745d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:20.923059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 597/1199] 10.222.0.21 () {56 vars in 1186 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/projects/db7b3552e97b48da836c2970506c75aa => generated 361 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:20.929505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:40:21.050568 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59874ac4-b1ed-43fa-b40a-c00f639e0886 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:40:21.086726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:40:21.087391 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:40:21.087679 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:40:21.087958 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:40:21.088821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=db7b3552e97b48da836c2970506c75aa, user.id=f9f6ee70b4ad45b0a49767a225f44571) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:40:21.089141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:40:21.090584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45501ceb-ba7e-44c1-a01d-f3afdcd55190 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:40:21.098090 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 603/1200] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:40:20 2022] GET /identity/v3/role_assignments?user.id=f9f6ee70b4ad45b0a49767a225f44571&scope.project.id=db7b3552e97b48da836c2970506c75aa&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:40:30.911706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=21 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:40:31.094244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=93 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:28.087897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05837b88-1e06-40df-87dd-924a679a9180 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:28.088874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05837b88-1e06-40df-87dd-924a679a9180 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:28.088874 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05837b88-1e06-40df-87dd-924a679a9180 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:28.090133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 598/1201] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:41:28.096968 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:28.097814 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:28.097814 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:28.118960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:28.120405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:28.251944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 604/1202] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:28.282029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05837b88-1e06-40df-87dd-924a679a9180 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:28.466352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05837b88-1e06-40df-87dd-924a679a9180 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:28.533919 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:28.535257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:28.535708 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:28.536421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:28.560255 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:28.564189 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:28.610727 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:28.611908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:28.613010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:28.613010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:28.613010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:28.613010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:28.613757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 599/1203] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:28.621800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:28.857606 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60222192-1f1f-49ad-ba60-f727fffb5616 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:28.932180 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:28.950634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:28.951365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:28.951657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:28.951944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:28.952727 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:28.953048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:28.954587 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59633696-cf79-4ce1-a80e-888d2f30ff4d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:28.977227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 605/1204] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:28.984885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:29.073632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:29.146742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54805c9-ba01-452d-9e1e-7f95d4aae5ae None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:29.216553 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:29.217324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:29.217734 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:29.218009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:29.218784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:29.220746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d151cb0-61a1-4afc-b10e-02a5bb0a3548 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:29.248835 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 600/1205] 10.222.0.21 () {60 vars in 1132 bytes} [Wed May 25 13:41:28 2022] POST /identity/v3/users => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:41:32.266832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02f9cc-a15d-49ca-933f-1db2a19885c5 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:32.267495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02f9cc-a15d-49ca-933f-1db2a19885c5 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:32.267495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02f9cc-a15d-49ca-933f-1db2a19885c5 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:32.268902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 606/1206] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:41:32.276196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:32.277367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:32.277367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:32.300744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:32.303466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:32.436769 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 601/1207] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:32.479459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02f9cc-a15d-49ca-933f-1db2a19885c5 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:32.608084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02f9cc-a15d-49ca-933f-1db2a19885c5 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:32.676019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:32.677111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:32.677437 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:32.677910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:32.702744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:32.714143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:32.716158 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:32.723713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 607/1208] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:32.724606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:32.790436 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:32.908732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7a157-f90d-4b9f-9081-de806585405f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:32.950275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:32.951059 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:32.951444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:32.951714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:32.952545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:32.952963 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:32.954389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:32.976058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 602/1209] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:32.983263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:33.062656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:33.157041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a6039-3f70-45bb-b506-4418a83ea9ea None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:33.209354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:33.210061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:33.210665 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:33.210665 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:33.224736 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:33.229850 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:33.252596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:33.253690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:33.254830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:33.254830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:33.254830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:33.254830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:33.254830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 608/1210] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:33.266306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:33.437563 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:33.442331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39c75-c3c3-444c-a966-c362ae5bd6b4 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:33.489547 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:33.490224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:33.490542 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:33.490851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:33.492196 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:33.492688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:33.494233 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:33.511163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 603/1211] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:33.518105 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:33.676585 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a2cbb-c8e3-402d-9891-4ac8e08d6c18 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:33.680942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:33.752756 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:33.754583 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:33.755082 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:33.755764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:33.770154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:33.771720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:41:33.790055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:33.792176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:33.793678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:33.798811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 609/1212] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:33.799164 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:33.931587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f75e9-10ee-4303-a789-619e3e0e9e1f None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:33.971198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:33.971908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:33.972181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:33.972556 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:33.973353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:33.973655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:33.975123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:33.999764 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 604/1213] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:41:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:34.013894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:34.147769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e101c56-94a8-46c2-b1a8-ff94e86b1f23 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:34.211622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:34.213026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:34.213548 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:34.213974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:34.272133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:34.334077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:34.336104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:34.504274 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 610/1214] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:41:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:41:34.512392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:34.552640 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:34.728737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b3f8e-0659-4792-bc14-3af7299ad214 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:34.805249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:34.806461 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:34.806895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:34.807306 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:34.827503 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:34.829615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:34.862191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:34.864030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:34.865933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:34.865933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:34.865933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:34.865933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:34.865933 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 605/1215] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:34.872640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:35.002631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:35.084744 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03c1cb-a7a5-472b-9b9e-b6caf4832aac None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:35.150197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:35.151352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:35.151822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:35.152260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:35.153356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:35.153856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:35.156061 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:35.174696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 611/1216] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:35.182634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:35.312717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:35.384851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549babf-f74a-4805-8c18-49c0d54c8398 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:35.454445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:35.455922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:35.456449 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:35.457018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:35.471981 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:35.475426 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:41:35.507475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:35.509743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:35.513755 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 606/1217] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:35.520738 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:35.683048 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a7c1a-a528-4a3f-b018-0bb3659c0c8e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:35.724408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:35.747229 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:35.748315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:35.748832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:35.749279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:35.750799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:35.751242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:35.754148 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d376a8c-202a-4050-aea7-4340d976cd36 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:35.777052 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 612/1218] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:41:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:35.787136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:35.990987 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35b3c8-7a0d-49a2-93c6-1d61f3c79bea None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:36.044918 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:36.056762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:36.057981 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:36.058558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:36.058991 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:36.060544 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=7280fc8f604e42cdb50973590acaab81) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:36.061156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:36.063877 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956cbce-21fc-41f4-a156-bf48cb2969df None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:36.080687 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 607/1219] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:41:35 2022] GET /identity/v3/role_assignments?user.id=7280fc8f604e42cdb50973590acaab81&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:39.001656 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ee09-dcce-48c7-a7cf-b86315cdcf55 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:39.002545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ee09-dcce-48c7-a7cf-b86315cdcf55 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:39.002902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ee09-dcce-48c7-a7cf-b86315cdcf55 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:39.005058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 613/1220] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:39.013743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:39.014371 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:39.014515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:39.035561 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:39.039876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:39.139126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 608/1221] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:39.183038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7ee09-dcce-48c7-a7cf-b86315cdcf55 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:39.329617 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7ee09-dcce-48c7-a7cf-b86315cdcf55 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:39.377500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:39.378732 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:39.379210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:39.379720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:39.395592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:39.405576 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:39.407569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:39.409459 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 614/1222] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:39.414185 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:39.547599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902654f-9ef4-404e-8c45-ed54ce01cb23 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:39.578450 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:39.603224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:39.604627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:39.605134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:39.605641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:39.607050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] RBAC: Adding query filter params (name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:39.607609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:39.610262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:39.627999 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 609/1223] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:41:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:39.634674 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:39.687452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:39.761616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6434a-fdd3-4d04-be23-9cfa0687f4cb None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:39.800227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:39.801082 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:39.801357 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:39.801637 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:39.816513 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:39.819389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:39.842586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:39.843819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:39.844817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:39.844817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:39.844817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:39.844817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:39.844817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 615/1224] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:39.852327 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:40.002166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f0a59-fefd-449d-87b4-ab3b4320a914 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:40.043767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:40.044661 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:40.044799 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:40.045039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:40.045803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:40.046116 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:40.047576 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:40.062441 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 610/1225] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:40.070393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:40.214081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:40.224670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa788f-34d4-4368-b103-c36a527c2ade None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:40.266775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:40.267690 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:40.268062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:40.268398 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:40.278014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:40.280440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:41:40.302737 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:40.305267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:40.307293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 616/1226] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:40.312928 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:40.407766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:40.500200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1552865-0d81-48b8-bca8-7a33443efe16 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:40.544404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:40.546397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:40.546397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:40.546710 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:40.548063 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:40.548630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:40.551333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:40.585178 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 611/1227] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:41:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:40.593475 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:40.728319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa76d-7780-4663-bc15-0e638051d265 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:40.787863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:40.789253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:40.789647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:40.790141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:40.807028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:40.809946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:40.847515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:40.849291 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:40.851356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:40.851356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:40.851356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:40.851356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:40.851356 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 617/1228] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:40.859376 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:41.019818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17bcd7-f273-4fcf-9ae6-98f73320e0d9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:41.023529 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:41.064115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:41.065137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:41.065137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:41.065459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:41.065927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:41.066485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:41.067897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:41.081979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 612/1229] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:41.090563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:41.105869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:41.247038 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a6e37-6ee9-4670-91ab-36c4b4cc4293 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:41.311700 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:41.313138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:41.313634 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:41.314190 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:41.328233 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:41.330812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:41:41.350724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:41.352893 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:41.354714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:41.359230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 618/1230] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:41.359545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:41.528361 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bed02-8ed2-429f-b588-aeaa0e2463a9 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:41.578183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:41.578805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:41.579091 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:41.579366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:41.580224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:41.580539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:41.582225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63a5c8-ba4b-4f77-b4f2-a41ef926596c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:41.606543 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 613/1231] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:41.615045 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:41.728430 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:41.781290 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01486b8-754e-467e-98e6-4eeae29bdc88 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:41.823405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:41.824338 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] REQUEST_METHOD: `PUT` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:41.824654 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:41.824976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/7280fc8f604e42cdb50973590acaab81/roles/3df30cf4e410480a890d14264879d404` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:41.843455 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=7280fc8f604e42cdb50973590acaab81, role_id=3df30cf4e410480a890d14264879d404)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:41.844853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244642cd-4252-4662-b085-be57f9bf48af None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:41.866293 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 619/1232] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:41:41 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/7280fc8f604e42cdb50973590acaab81/roles/3df30cf4e410480a890d14264879d404 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:41:44.738126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568a8e-a53f-4f0f-9c74-d56d1069aa20 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:44.738774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568a8e-a53f-4f0f-9c74-d56d1069aa20 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:44.738774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568a8e-a53f-4f0f-9c74-d56d1069aa20 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:44.740375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 614/1233] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:41:44.748092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:44.748712 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:44.748939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:44.769066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:44.770561 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:44.860652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 620/1234] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:44.884468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568a8e-a53f-4f0f-9c74-d56d1069aa20 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:44.917208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:45.029705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568a8e-a53f-4f0f-9c74-d56d1069aa20 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:45.070368 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:45.071404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:45.071778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:45.072153 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:45.085960 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:45.096304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:45.098229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:45.100466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 615/1235] 10.222.0.21 () {56 vars in 1102 bytes} [Wed May 25 13:41:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:45.106013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:45.177016 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:45.257536 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899827e8-16e2-4b53-bc43-3638c18892a3 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:45.305622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:45.306316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:45.306665 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:45.306870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:45.308015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] RBAC: Adding query filter params (name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:45.308460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:45.309948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:45.320590 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 621/1236] 10.222.0.21 () {56 vars in 1103 bytes} [Wed May 25 13:41:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:45.327042 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:45.433312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:45.450587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a68aa6-34b0-416f-9ff0-e56fe23e7e49 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:45.488136 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:45.488979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:45.489262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:45.489535 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:45.502588 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:45.504191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:45.530010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:45.532140 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:45.534183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:45.534183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:45.534183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:45.534183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:45.534183 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 616/1237] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:45.538822 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:45.694267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2eae74-1f97-4fef-98f8-47e224c6ed7b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:45.705830 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:45.752454 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78766-016c-41e1-9308-db810d15252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:45.753545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78766-016c-41e1-9308-db810d15252d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:45.754013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78766-016c-41e1-9308-db810d15252d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:45.754463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78766-016c-41e1-9308-db810d15252d None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:45.755345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78766-016c-41e1-9308-db810d15252d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:45.755907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78766-016c-41e1-9308-db810d15252d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:45.757558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78766-016c-41e1-9308-db810d15252d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:45.774897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 622/1238] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:45.780621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:45.901773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad22fc-f8d3-4fdd-a813-1e2e205a6daf None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:45.945188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:45.946085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:45.946467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:45.946851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:45.950767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:45.957910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:45.960188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:41:45.978485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:45.980783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:45.982941 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 617/1239] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:45.988721 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78766-016c-41e1-9308-db810d15252d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:46.129448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78766-016c-41e1-9308-db810d15252d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:46.191800 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:46.192657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:46.193015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:46.193393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:46.194392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:46.194771 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:46.197022 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:46.221694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 623/1240] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:41:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:46.230068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:46.410341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38709a-f559-4154-a8fd-396245b0c5e0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:46.469044 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:46.469910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:46.470369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:46.470931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:46.556429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:46.569608 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:46.578194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:46.748739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 618/1241] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:41:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:41:46.757427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:46.800439 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:46.927725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ad74-7d11-4982-95fa-b15757d2b2fb None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:46.979549 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:46.980371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:46.980694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:46.980975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:46.995189 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:46.997938 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:47.022414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:47.024042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:47.025279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:47.025279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:47.025279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:47.025279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:47.025279 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 624/1242] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:47.030977 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:47.150694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:47.157996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f0ed-91f2-4004-95a9-27d015a91ba1 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:47.211020 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:47.212194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:47.212789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:47.213262 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:47.214321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:47.214785 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:47.216983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:47.231240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 619/1243] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:47.237622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:47.341164 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:47.379032 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29392-f6e8-4319-b5cc-72e9101cb937 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:47.440314 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:47.441594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:47.441984 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:47.442511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:47.453544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:47.455708 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:41:47.475609 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:47.476862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:47.477918 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 625/1244] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:41:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:47.483671 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:47.646988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec12718-05de-4431-89cb-96960a5dd956 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:47.715352 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:47.716632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:47.717145 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:47.717651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:47.719204 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:47.719767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:47.722431 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4054c22-951d-43df-9f67-dd3cf4db4167 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:47.749648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 620/1245] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:41:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:47.760887 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:47.906078 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9bfb5-98f9-43ca-aefc-a49095242ea0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:47.947832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:47.949519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:47.949519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:47.949519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:47.949884 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] RBAC: Adding query filter params (role.id=3df30cf4e410480a890d14264879d404, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=7280fc8f604e42cdb50973590acaab81) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:47.950192 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:47.951631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df35bc-14d6-4ab4-b241-be17f5c08a24 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:47.958563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 626/1246] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:41:47 2022] GET /identity/v3/role_assignments?user.id=7280fc8f604e42cdb50973590acaab81&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=3df30cf4e410480a890d14264879d404 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:50.703808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4b80-0594-4422-92c1-aff726fe75e0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:50.704717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4b80-0594-4422-92c1-aff726fe75e0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:50.704717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c4b80-0594-4422-92c1-aff726fe75e0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:50.705756 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 621/1247] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:50.714316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:50.714932 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:50.715337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:50.723332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:50.741186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:50.742566 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:50.839102 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 627/1248] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:50.881885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4b80-0594-4422-92c1-aff726fe75e0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:50.919031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:51.051247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c4b80-0594-4422-92c1-aff726fe75e0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:51.098782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:51.100142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:51.100714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:51.100970 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:51.116704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:51.127244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:51.129252 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:51.130850 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 622/1249] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:51.137824 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:51.278907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f1f9-f812-44b5-a7cf-f0c3fe311a97 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:51.321982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:51.330040 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:51.331125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:51.331603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:51.331999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:51.333924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:51.334678 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:51.337717 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:51.356836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 628/1250] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:51.363506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:51.485052 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:51.520751 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6de471-8a74-45cd-8f3b-0d1814f74c1e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:51.578208 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:51.579254 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:51.579742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:51.580207 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:51.602434 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:51.604492 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:51.627388 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:51.629550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:51.631494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:51.631494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:51.631494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:51.631494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:51.631494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 623/1251] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:51.638720 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:51.829770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5e790-d32c-4b00-bf8f-b6cfb18ea68c None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:51.900539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:51.901875 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:51.902382 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:51.902907 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:51.904593 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:51.905153 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:51.908218 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:51.934265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 629/1252] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:51.941142 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:52.084659 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e3dca-b785-45be-850e-9541d4d4aa41 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:52.115891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:52.125862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:52.126688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:52.126976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:52.127258 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:52.136939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:52.138448 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:41:52.154331 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:52.156272 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:52.158121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 624/1253] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:52.162322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:52.231013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:52.288889 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e77fe-f231-4157-82cd-3916fc48d54b None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:52.331802 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:52.332813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:52.333784 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:52.334213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:52.335574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:52.335828 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:52.337575 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:52.368000 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 630/1254] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:41:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:52.379080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:52.553908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4e77f-8afa-4c91-9068-a27412ab9563 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:52.612853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:52.613959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:52.614506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:52.615193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:52.714748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:52.738512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:52.743367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:52.911771 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 625/1255] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:41:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:41:52.917748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:53.072587 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3ed01-0442-4524-bef4-99a4d909566e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:53.092863 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:53.119044 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:53.120009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:53.120385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:53.120786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:53.133611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:53.135427 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:53.174265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:53.175462 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:53.177303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:53.177303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:53.177303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:53.177303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:53.177303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 631/1256] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:53.185789 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:53.309964 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:53.327562 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982ea80-4eb7-4ff4-b257-e110a03f48ae None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:53.397831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:53.399134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:53.399763 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:53.400340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:53.401974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:53.402628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:53.405524 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:53.429655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 626/1257] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:53.436137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:53.593739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445c880-18c5-430b-b16d-0b75d6714ea3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:53.649324 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:53.650461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:53.650823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:53.651201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:53.661578 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:53.664120 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:41:53.685900 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:53.687227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:41:53.688725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 632/1258] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:53.696061 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:53.836942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194ab98-724b-40db-882c-12fe1540d70e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:53.885552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:53.886309 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:53.886750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:53.887085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:53.887973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:53.888367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:53.890316 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff40e-6037-4ca6-8ac1-458e3330a855 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:53.905642 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 627/1259] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:53.913953 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:53.952770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:54.085194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf50f-1545-4646-ab7d-a5ceff96f8c9 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:54.135416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:54.136222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:54.136595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:54.137079 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:54.138111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=7280fc8f604e42cdb50973590acaab81) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:54.138535 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:54.140143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d803a5-6c20-4f56-88fb-5b8724666172 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:54.147547 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 633/1260] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:41:53 2022] GET /identity/v3/role_assignments?user.id=7280fc8f604e42cdb50973590acaab81&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:56.552731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42015-e957-46ad-a64c-47611d8100e2 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:56.553430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42015-e957-46ad-a64c-47611d8100e2 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:56.553911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d42015-e957-46ad-a64c-47611d8100e2 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:56.556354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 628/1261] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:41:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:41:56.562254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:56.562798 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:56.562798 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:56.580519 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:41:56.581787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:41:56.656642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 634/1262] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:41:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:41:56.680864 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d42015-e957-46ad-a64c-47611d8100e2 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:56.694958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:56.816876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d42015-e957-46ad-a64c-47611d8100e2 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:56.862086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:56.862910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:56.863202 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:56.863483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:56.875463 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:41:56.884086 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:56.885378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:56.886425 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 629/1263] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:41:56.896472 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:56.930839 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:57.078384 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952f79c-d76a-4496-af37-a2d4a336c9bf None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:57.122058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:57.122755 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:57.123043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:57.123326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:57.124636 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:57.125084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:57.127005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:57.140100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 635/1264] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:41:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:57.148976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:57.227882 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:57.265169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159bcd9-087f-41f2-b98f-be135125d09e None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:57.300483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:57.301975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:57.302641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:57.302924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:57.314347 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:57.316087 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:57.334624 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:57.342156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:57.343247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:57.343247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:57.343247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:57.343247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:57.343247 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 630/1265] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:57.343762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:57.456760 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:57.477313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105fa3-cf53-4c80-9a31-8405c174e412 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:57.524254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:57.524971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:57.525262 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:57.525544 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:57.526294 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:57.526610 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:57.528034 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:57.542967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 636/1266] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:57.549030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:57.668216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc69e1-8e6f-47d9-b742-cc05784415db None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:57.689187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:57.703883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:57.704739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:57.705031 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:57.705311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:57.714127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:57.715607 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:41:57.731338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:57.733267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:57.735437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 631/1267] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:57.740753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:57.845057 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2bb48-9f29-428e-86ba-fa53b1348d23 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:57.882728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:57.883332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:57.883614 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:57.883892 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:57.884869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:57.885183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:57.886694 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:57.897836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:57.907567 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 637/1268] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:41:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:57.914205 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:58.024531 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882e633-e9a6-489b-ad96-b47e9298a3b0 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:58.061421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:58.062219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:58.062505 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:58.062783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:58.074672 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:58.076143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:41:58.094627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:58.096433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:58.098827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:41:58.098827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:41:58.098827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:41:58.098827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:58.098827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 632/1269] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:58.105326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:58.206362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9b3c3-2ebe-4480-9e47-34429a41ed98 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:58.241577 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:58.242242 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:58.242529 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:58.242809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:58.243531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:58.243841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:58.245248 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:58.257089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 638/1270] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:41:58.262971 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:58.295451 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:58.372646 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12305d3-c22c-44e2-b018-51f68e022152 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:58.408219 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:58.409046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:58.409335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:58.409615 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:58.417048 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:58.418509 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:41:58.432537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:41:58.433701 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:41:58.434693 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 633/1271] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:41:58.439971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:58.505752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:58.557471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fe325-c4ba-4e4c-937c-3987dabe8bd1 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:58.593375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:58.594042 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:58.594329 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:58.594608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:58.595414 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:41:58.595724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:58.597215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0467e7-2c92-4a2c-90e5-7b2628f836a9 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:58.611388 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 639/1272] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:41:58.618097 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:41:58.716643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:41:58.728854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00c2f9-1f0f-4d9b-9018-ae1ca9d4b68a None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:41:58.768060 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:41:58.768894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:41:58.769177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:41:58.769451 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] PATH_INFO: `/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/7280fc8f604e42cdb50973590acaab81/roles/698416d22e5b4c278ff780c5eef36ede` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:41:58.772069 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] RBAC: Authorizing `identity:create_grant(project_id=9f61eee6bc284eb2b31fa2d98e2ff62e, user_id=7280fc8f604e42cdb50973590acaab81, role_id=698416d22e5b4c278ff780c5eef36ede)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:41:58.773558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e1dbd-7842-4aa6-9f88-16dfa61c732d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:41:58.789082 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 634/1273] 10.222.0.21 () {58 vars in 1439 bytes} [Wed May 25 13:41:58 2022] PUT /identity/v3/projects/9f61eee6bc284eb2b31fa2d98e2ff62e/users/7280fc8f604e42cdb50973590acaab81/roles/698416d22e5b4c278ff780c5eef36ede => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:42:01.122885 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9279e8-36e4-4a6c-b64e-fddfd02c140d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:01.123463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9279e8-36e4-4a6c-b64e-fddfd02c140d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:01.123463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9279e8-36e4-4a6c-b64e-fddfd02c140d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:01.124967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 640/1274] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:01.131432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:01.132104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:01.132324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:01.155281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:01.156515 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:01.266344 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 635/1275] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:42:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 13:42:01.296779 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9279e8-36e4-4a6c-b64e-fddfd02c140d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:01.432330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9279e8-36e4-4a6c-b64e-fddfd02c140d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:01.459077 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:01.479379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:01.480184 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:01.480476 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:01.480787 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:01.494091 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 13:42:01.501855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:01.503595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:01.505520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 641/1276] 10.222.0.21 () {56 vars in 1096 bytes} [Wed May 25 13:42:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:01.509867 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:01.606788 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:01.624893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d6442-9162-41a3-b738-0fa205f30041 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:01.658483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:01.659204 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:01.659530 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:01.659848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:01.661318 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:01.661666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:01.663104 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:01.672443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 636/1277] 10.222.0.21 () {56 vars in 1099 bytes} [Wed May 25 13:42:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:01.678668 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:01.791705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884283-0296-4e31-ac10-078bd11ee788 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:01.837439 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:01.838256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:01.838543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:01.838821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:01.851139 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:01.852631 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:01.870596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:01.871724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:01.872841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:01.872841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:01.872841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:01.872841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:01.872841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 642/1278] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:01.877965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:01.997665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071c52c-1313-4202-ac9e-8fa79dfcae12 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:02.038583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:02.039723 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:02.040235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:02.040796 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] PATH_INFO: `/v3/domains` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:02.042169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:02.042647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:02.045267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:02.058389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 637/1279] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:02.064182 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:02.097506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:02.167905 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf10864-bdb8-40e0-b3b5-ce6ce52e932e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:02.202968 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:02.203771 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:02.204053 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:02.204328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:02.212463 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:02.213959 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 13:42:02.228948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:02.230175 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:02.231507 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 643/1280] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:42:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:02.236908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:02.242056 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:02.350745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc79fa-3f76-4e96-8add-e142e1f71409 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:02.388854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:02.389533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:02.389816 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:02.390089 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] PATH_INFO: `/v3/users` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:02.390928 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:02.391238 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:02.392706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:02.411422 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 638/1281] 10.222.0.21 () {56 vars in 1143 bytes} [Wed May 25 13:42:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:02.417264 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:02.511371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:02.521855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536f7b0-fc53-4bdb-91d0-57a6b8738196 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:02.558275 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:02.558934 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:02.559220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:02.559497 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:02.635362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:02.636790 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:02.739819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:02.750770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 644/1282] 10.222.0.21 () {58 vars in 1282 bytes} [Wed May 25 13:42:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 13:42:02.757271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:02.824765 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:02.866773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695abd0c-252a-4a9c-accf-54e8b34749be None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:02.906410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:02.907220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:02.907500 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:02.907776 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:02.919486 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:02.921029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 13:42:02.940144 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:02.941442 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:02.942641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 13:42:02.942641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 13:42:02.942641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 13:42:02.942641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:42:02.942641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 639/1283] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 13:42:02.947508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:03.063970 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d340d8-fd3c-4de8-b26c-636f0baa108e None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:03.104841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:03.105534 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:03.105823 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:03.106103 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:03.106855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:03.107169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:03.108635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:03.121853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 645/1284] 10.222.0.21 () {56 vars in 1109 bytes} [Wed May 25 13:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:03.128161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:03.242036 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54db2e-2048-4127-8c27-3c7f4253f0ca None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:03.260366 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:03.286206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:03.287037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:03.287326 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:03.287600 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:03.295589 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:03.297133 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 13:42:03.313297 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application with Lock( May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:03.314714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: ERROR keystone.server.flask.application May 25 13:42:03.315662 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 640/1285] 10.222.0.21 () {56 vars in 1111 bytes} [Wed May 25 13:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:03.322176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:03.378471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:03.431449 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e463dc5-5e5e-4e4b-b3ad-b21b7d0c1c7d None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:03.467550 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:03.468215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:03.468504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:03.468815 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:03.469611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:03.469924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:03.471379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798dd95-6358-449e-8017-cca41cd2b26b None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:03.484379 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 646/1286] 10.222.0.21 () {56 vars in 1148 bytes} [Wed May 25 13:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 13:42:03.490976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:03.598088 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84356592-a2b1-4d7d-9ccf-0257fb8e8181 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:03.640026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:03.640731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:03.641024 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:03.641304 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:03.642161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] RBAC: Adding query filter params (role.id=698416d22e5b4c278ff780c5eef36ede, scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e, user.id=7280fc8f604e42cdb50973590acaab81) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:03.642478 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:03.644793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c6a67-a8ce-4dbd-8250-3b75d259a982 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:03.655800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 641/1287] 10.222.0.21 () {56 vars in 1374 bytes} [Wed May 25 13:42:03 2022] GET /identity/v3/role_assignments?user.id=7280fc8f604e42cdb50973590acaab81&scope.project.id=9f61eee6bc284eb2b31fa2d98e2ff62e&role.id=698416d22e5b4c278ff780c5eef36ede => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:06.032009 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065261b-70dd-4671-bcd5-129848e5b668 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:06.032624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065261b-70dd-4671-bcd5-129848e5b668 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:06.032624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065261b-70dd-4671-bcd5-129848e5b668 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:06.033975 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 647/1288] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:06.039731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:06.040246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:06.040410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:06.058239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:06.059466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:06.092333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:06.132124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 642/1289] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:42:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:06.155041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065261b-70dd-4671-bcd5-129848e5b668 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:06.262114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065261b-70dd-4671-bcd5-129848e5b668 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:06.273766 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:06.298504 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:06.299221 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:06.299508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:06.299789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:06.301027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:06.302412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:06.314616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:06.315906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:06.317111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 648/1290] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:42:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:06.323330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:06.443404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa993e-b4e1-453f-a435-602da77c4aea None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:06.481811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:06.482480 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:06.482762 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:06.483038 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:06.483749 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:06.484054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:06.485470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2b249-2cc9-4e4f-8c0e-f0de5f501cc8 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:06.491558 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 643/1291] 10.222.0.21 () {56 vars in 1116 bytes} [Wed May 25 13:42:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:06.497491 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:06.606084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60124ea-bc18-4121-8f8b-d2e23507e837 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:06.645287 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:06.645971 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:06.646261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:06.646543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] PATH_INFO: `/v3/services` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:06.647263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] RBAC: Adding query filter params (type=placement) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:06.647575 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:06.649051 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc558454-c249-46ad-8f8e-b6f5cdaefcdb None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:06.655055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 649/1292] 10.222.0.21 () {56 vars in 1116 bytes} [Wed May 25 13:42:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:08.920455 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d201da-a208-4889-8d3e-84f117adb8a7 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:08.921039 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d201da-a208-4889-8d3e-84f117adb8a7 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:08.921155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d201da-a208-4889-8d3e-84f117adb8a7 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:08.923249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 644/1293] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:42:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:08.931955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c0dfb-4599-40c7-ad5c-7f7a8d7680e0 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:08.932511 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c0dfb-4599-40c7-ad5c-7f7a8d7680e0 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:08.932949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c0dfb-4599-40c7-ad5c-7f7a8d7680e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:08.960904 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-6b0c0dfb-4599-40c7-ad5c-7f7a8d7680e0 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:08.962935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-6b0c0dfb-4599-40c7-ad5c-7f7a8d7680e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:09.007948 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:09.051627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 650/1294] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:42:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:09.075215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d201da-a208-4889-8d3e-84f117adb8a7 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:09.108831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:09.194417 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d201da-a208-4889-8d3e-84f117adb8a7 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:09.234998 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:09.235677 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:09.235979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:09.236240 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:09.237011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:09.238498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab43c9-22b4-4863-9c43-e471fe625921 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:09.253848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 645/1295] 10.222.0.21 () {60 vars in 1141 bytes} [Wed May 25 13:42:09 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 13:42:11.545332 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789067e-3a22-4d91-9ff8-67cc9ca8938f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:11.545881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789067e-3a22-4d91-9ff8-67cc9ca8938f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:11.545881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789067e-3a22-4d91-9ff8-67cc9ca8938f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:11.547254 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 651/1296] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:11.553156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:11.553673 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:11.553766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:11.572698 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:11.573932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:11.651728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 646/1297] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:42:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:11.675974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789067e-3a22-4d91-9ff8-67cc9ca8938f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:11.787011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789067e-3a22-4d91-9ff8-67cc9ca8938f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:11.823499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:11.824247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:11.824534 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:11.824852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:11.825528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:11.826991 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:11.840697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:11.842243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:11.843858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 652/1298] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:42:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:11.849378 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:11.882914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:11.979054 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f056df-4818-41ec-86c5-0e50d38096cd None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.014922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.015594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.015876 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.016150 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.016890 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:12.017197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.018634 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.024296 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 647/1299] 10.222.0.21 () {56 vars in 1116 bytes} [Wed May 25 13:42:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:12.030602 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:12.063084 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:12.148493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda479c-bb41-43f6-a19d-9220808675c0 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.183374 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.184037 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.184322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.184632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.185432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] RBAC: Adding query filter params (interface=public, service_id=950a185c1bd94ddb9b4f6c37c3a18b10, region_id=RegionOne) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:12.185751 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.187132 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a958316-a5d7-4088-b1b1-0af3e1c8803c None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.193349 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 653/1300] 10.222.0.21 () {56 vars in 1251 bytes} [Wed May 25 13:42:12 2022] GET /identity/v3/endpoints?service_id=950a185c1bd94ddb9b4f6c37c3a18b10&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:12.199141 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:12.283072 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:12.303222 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edda8e7-7227-49a0-b353-878f965fa300 None admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:12.338508 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:12.339330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:12.339620 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:12.339898 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:12.340625 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:12.342628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1aaf89-9d03-43de-9bfe-0e9dd6e83f1c None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:12.354880 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 648/1301] 10.222.0.21 () {56 vars in 1087 bytes} [Wed May 25 13:42:12 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:42:14.649775 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfaa1-0696-4d5e-b3af-694fa4c9fb6d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:14.650298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfaa1-0696-4d5e-b3af-694fa4c9fb6d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:14.650418 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cfaa1-0696-4d5e-b3af-694fa4c9fb6d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:14.651693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 654/1302] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:42:14.656891 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:14.657319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:14.657445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:14.675585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:42:14.676955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:42:14.743437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 649/1303] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:42:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 13:42:14.767225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cfaa1-0696-4d5e-b3af-694fa4c9fb6d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:14.874024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:14.882232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cfaa1-0696-4d5e-b3af-694fa4c9fb6d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:14.917103 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:14.917910 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:14.918198 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:14.918480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:14.919133 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:14.920508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 13:42:14.932551 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 13:42:14.933776 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.get_or_create( May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application with Lock( May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self._enter() May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application return self.creator() May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application created_value = creator( May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: ERROR keystone.server.flask.application May 25 13:42:14.934724 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 655/1304] 10.222.0.21 () {56 vars in 1117 bytes} [Wed May 25 13:42:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 13:42:14.940307 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.019837 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:15.043155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02856ee-7ed6-4674-9f3e-e379178c524c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.078707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.079368 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.079650 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.079925 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] PATH_INFO: `/v3/services` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.081161 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] RBAC: Adding query filter params (name=placement) {{(pid=91826) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:42:15.081490 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:15.082846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ef15f-efa4-4074-b139-cff844be569d None admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:15.088129 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 650/1305] 10.222.0.21 () {56 vars in 1116 bytes} [Wed May 25 13:42:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:42:15.093902 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:42:15.204888 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a2d55-d856-484e-99c1-e8b9b50936b3 None admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:42:15.240223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:42:15.240920 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:42:15.241209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:42:15.241488 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:42:15.242227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:42:15.243598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3bada-aaaf-421f-8755-4addc14119b3 None admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:42:15.262669 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 656/1306] 10.222.0.21 () {60 vars in 1144 bytes} [Wed May 25 13:42:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:42:25.085191 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=30 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:42:25.258129 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:21.965983 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062928-a626-422a-9ef1-e473c13b5bae None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:21.966980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062928-a626-422a-9ef1-e473c13b5bae None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:21.966980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79062928-a626-422a-9ef1-e473c13b5bae None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:21.977783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 651/1307] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:21.979118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353b843-978e-41d9-916a-cbb25438c807 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:21.979118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353b843-978e-41d9-916a-cbb25438c807 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:21.979118 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353b843-978e-41d9-916a-cbb25438c807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:22.189354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-6353b843-978e-41d9-916a-cbb25438c807 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:22.190603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-6353b843-978e-41d9-916a-cbb25438c807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:22.384586 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 657/1308] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:22.423093 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:22.423623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:22.423911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:22.425746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 652/1309] 10.222.0.21 () {54 vars in 964 bytes} [Wed May 25 13:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:22.431405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666407-b1bb-44ab-91ff-6eb5f623e990 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:22.431699 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666407-b1bb-44ab-91ff-6eb5f623e990 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:22.432005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7666407-b1bb-44ab-91ff-6eb5f623e990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:22.517935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-a7666407-b1bb-44ab-91ff-6eb5f623e990 None None] MFA Rules not processed for user `6d754f1173e446ceb607dfaeb2231c28`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:22.519172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-a7666407-b1bb-44ab-91ff-6eb5f623e990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:22.542581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:22.625873 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 658/1310] 10.222.0.21 () {58 vars in 1061 bytes} [Wed May 25 13:43:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:22.626550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:22.628351 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:22.768226 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f899c-bc97-4718-b7cf-598aa35fd211 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:22.849818 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:22.850597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:22.850939 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:22.851322 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:22.855599 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:22.920229 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:22.981996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:22.983475 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f0364-d838-4ee6-b944-45cd4513d6e8 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:23.004665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 653/1311] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:43:25.668601 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:25.669156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:25.669156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:25.670687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 659/1312] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:25.676073 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7334f7-e4ba-4935-8ad7-ae514c78dcb6 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:25.676485 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7334f7-e4ba-4935-8ad7-ae514c78dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:25.676803 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7334f7-e4ba-4935-8ad7-ae514c78dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:25.748110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-bd7334f7-e4ba-4935-8ad7-ae514c78dcb6 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:25.749356 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-bd7334f7-e4ba-4935-8ad7-ae514c78dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:25.856438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 654/1313] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:43:25.904869 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:25.906624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:26.013340 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c55b73-5104-449b-9d42-826bf07ae30b None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:26.088387 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:26.091415 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:26.092321 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:26.092646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:26.092981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:26.095067 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:26.207648 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:26.209152 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4423a-8c9a-4498-b728-5e3a15cc7e04 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:26.228995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 660/1314] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:28.811021 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:28.811586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:28.811586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:28.813009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 655/1315] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:28.818197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1d15-c61e-4e54-8005-6016d418f643 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:28.818611 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1d15-c61e-4e54-8005-6016d418f643 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:28.818764 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa1d15-c61e-4e54-8005-6016d418f643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:28.882375 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-22fa1d15-c61e-4e54-8005-6016d418f643 None None] MFA Rules not processed for user `fc1945641a034317ac1e53f84a83c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:28.883498 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-22fa1d15-c61e-4e54-8005-6016d418f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:28.935230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:28.980820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 661/1316] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:29.019606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:29.021206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:29.031481 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:29.131210 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ffb4-a61b-44d5-9803-ecdc3b1582f2 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:29.206482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:29.207132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:29.207419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:29.207700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:29.209379 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:29.271910 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:29.320355 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:29.321851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a022b91-1635-425a-9413-84dc979b49e5 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:29.341409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 656/1317] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:32.183268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:32.183832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:32.183832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:32.185228 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 662/1318] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:32.190468 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d03-d56f-4fd7-9a57-9a311a096e63 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:32.190881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d03-d56f-4fd7-9a57-9a311a096e63 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:32.191037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb01d03-d56f-4fd7-9a57-9a311a096e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:32.255032 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-7cb01d03-d56f-4fd7-9a57-9a311a096e63 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:32.256255 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-7cb01d03-d56f-4fd7-9a57-9a311a096e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:32.350623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 657/1319] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:32.389670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:32.391191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:32.492015 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef90fab-41e2-40b7-b49d-5676749fa125 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:32.565876 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:32.566521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:32.566804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:32.567081 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:32.568732 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:32.575626 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:32.677808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:32.679311 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26263af-f3a3-420f-b932-d315fb9f876d service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:32.698503 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 663/1320] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:35.656866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:35.657465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:35.657465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:35.658817 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 658/1321] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:35.664176 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cb5e-3281-4ea1-bb7c-17721e8f8266 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:35.664651 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cb5e-3281-4ea1-bb7c-17721e8f8266 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:35.664777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71cb5e-3281-4ea1-bb7c-17721e8f8266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:35.732878 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-6b71cb5e-3281-4ea1-bb7c-17721e8f8266 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:35.734075 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-6b71cb5e-3281-4ea1-bb7c-17721e8f8266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:35.792392 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:35.834004 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 664/1322] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:35.870667 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:35.872250 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:35.894927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:35.984757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b490785-dd7a-415a-9594-bcd4cea6776c None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:36.060338 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:36.061032 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:36.061464 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:36.061780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:36.063445 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:36.142571 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:36.181639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:36.183342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078a174-b8d2-4227-bc14-80d3c5282eec service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:36.201534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 659/1323] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:43:38.556592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee54be2-15db-4df4-b921-5b5e25df152a None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:38.557460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee54be2-15db-4df4-b921-5b5e25df152a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:38.557460 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee54be2-15db-4df4-b921-5b5e25df152a None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:38.558867 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 665/1324] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:38.565108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1897-2b6b-4844-86f8-2dc0b2a222a9 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:38.565633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1897-2b6b-4844-86f8-2dc0b2a222a9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:38.565748 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda1897-2b6b-4844-86f8-2dc0b2a222a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:38.642030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-8fda1897-2b6b-4844-86f8-2dc0b2a222a9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:38.643249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-8fda1897-2b6b-4844-86f8-2dc0b2a222a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:38.741806 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 660/1325] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:38.793639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:38.793936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:38.794215 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:38.795594 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 666/1326] 10.222.0.21 () {54 vars in 964 bytes} [Wed May 25 13:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:38.802162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bacfb-0084-4392-9757-e911748ba135 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:38.802453 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bacfb-0084-4392-9757-e911748ba135 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:38.802732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bacfb-0084-4392-9757-e911748ba135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:38.832849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:38.880523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-dd3bacfb-0084-4392-9757-e911748ba135 None None] MFA Rules not processed for user `6d754f1173e446ceb607dfaeb2231c28`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:38.881744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-dd3bacfb-0084-4392-9757-e911748ba135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:38.962275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 661/1327] 10.222.0.21 () {58 vars in 1061 bytes} [Wed May 25 13:43:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:43:38.971657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:38.973240 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:39.091879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db78139-bc69-45e5-bf96-ba95316b3b93 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:39.172897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:39.173558 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:39.173844 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:39.174123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:39.175820 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:39.193819 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:39.308964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:39.310428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee816b38-ad62-4a1a-8cab-96b0295b5889 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:39.330569 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 667/1328] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:41.869393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:41.869937 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:41.869937 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:41.871387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 662/1329] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:41.877062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d095-fdcb-4e45-94ea-7bca04d89dd2 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:41.877518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d095-fdcb-4e45-94ea-7bca04d89dd2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:41.877627 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36d095-fdcb-4e45-94ea-7bca04d89dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:41.951713 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-cd36d095-fdcb-4e45-94ea-7bca04d89dd2 None None] MFA Rules not processed for user `fc1945641a034317ac1e53f84a83c2a2`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:41.953799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-cd36d095-fdcb-4e45-94ea-7bca04d89dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:42.021702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:42.053941 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 668/1330] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:42.092216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:42.093782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:42.201581 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967563a4-8c7e-496c-b69c-532b8aa0d99b None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:42.222725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:42.278023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:42.278674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:42.278959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:42.279234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:42.280915 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:42.397199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:42.398714 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb4f20-5816-4295-b615-99bf13cf9905 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:42.416841 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 663/1331] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:45.090995 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:45.091563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:45.091563 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:45.092992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 669/1332] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:45.098410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3287-7f4f-4d60-adae-22b409603f69 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:45.098866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3287-7f4f-4d60-adae-22b409603f69 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:45.098965 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a3287-7f4f-4d60-adae-22b409603f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:45.165151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:45.168201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-c45a3287-7f4f-4d60-adae-22b409603f69 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:45.169578 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-c45a3287-7f4f-4d60-adae-22b409603f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:45.269772 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 664/1333] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:45.306706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:45.308253 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:45.414604 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9e000-d898-4f2d-b6f5-3cf13f0544d8 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:45.493396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:45.494409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:45.494696 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:45.494973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:45.496628 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:45.525433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:45.612007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:45.613484 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63327618-2590-44c1-aba9-938b439f19a9 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:45.632125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 670/1334] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:47.967010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:47.967536 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:47.967630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:47.969018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 665/1335] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:47.974408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff38a-cd9f-42c1-b383-eed96deb5e15 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:47.974857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff38a-cd9f-42c1-b383-eed96deb5e15 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:47.974951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff38a-cd9f-42c1-b383-eed96deb5e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:48.047929 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-1f5ff38a-cd9f-42c1-b383-eed96deb5e15 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:48.049187 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-1f5ff38a-cd9f-42c1-b383-eed96deb5e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:48.123445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:48.145954 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 671/1336] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:48.186757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:48.188312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:48.292026 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d5cf9-f22e-40c9-a31a-3614964c4014 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:48.322856 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:48.391200 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:48.391773 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:48.392062 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:48.392337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:48.394079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:48.504815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:48.506283 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c46a87-4d10-4520-9165-374d080b64cd service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:48.524580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 666/1337] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:51.278471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:51.279049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:51.279049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:51.280378 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 672/1338] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:51.285689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d80d8-a124-41c7-a481-36e75e715974 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:51.286172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d80d8-a124-41c7-a481-36e75e715974 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:51.286283 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269d80d8-a124-41c7-a481-36e75e715974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:51.358638 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-269d80d8-a124-41c7-a481-36e75e715974 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:51.359853 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-269d80d8-a124-41c7-a481-36e75e715974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:51.369271 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:51.472840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 667/1339] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:51.509443 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:51.510981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:51.619429 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5505ab-2de3-48e1-9882-b24d834fb30c None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:51.694278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:51.695225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:51.695518 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:51.695796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:51.697495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:51.734305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:51.810841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:51.812698 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285cc03-a325-4f9c-b8a9-e436631aa332 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:51.832880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 673/1340] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:43:56.267647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:56.268702 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:56.269188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:56.275409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 668/1341] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:56.276906 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d998-b343-4eed-aa43-0a51232e77cb None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:56.277362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d998-b343-4eed-aa43-0a51232e77cb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:56.277453 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9d998-b343-4eed-aa43-0a51232e77cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:56.357753 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-90b9d998-b343-4eed-aa43-0a51232e77cb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:56.359312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-90b9d998-b343-4eed-aa43-0a51232e77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:56.454675 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:56.468581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 674/1342] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:43:56.508203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:56.510350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:56.610010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e52c6-d4ae-4eb1-a7f4-8838a1a0da22 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:56.613630 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:56.614279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:56.614564 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:56.614848 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:56.624775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:56.626220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1dae71-36c4-47ce-9a22-a21aabc10c1b service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:56.636700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 669/1343] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:43:59.478697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129986-cd85-4486-a2ed-0279448f6638 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:59.479225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129986-cd85-4486-a2ed-0279448f6638 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:59.479225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d129986-cd85-4486-a2ed-0279448f6638 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:59.480520 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 675/1344] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:43:59.485917 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269776f7-6fd8-4914-8614-7bd4c93b20c4 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:59.486372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269776f7-6fd8-4914-8614-7bd4c93b20c4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:59.486545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269776f7-6fd8-4914-8614-7bd4c93b20c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:59.514901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-269776f7-6fd8-4914-8614-7bd4c93b20c4 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:43:59.516085 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-269776f7-6fd8-4914-8614-7bd4c93b20c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:43:59.545076 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:43:59.599838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 670/1345] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:43:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:43:59.637172 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d129986-cd85-4486-a2ed-0279448f6638 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:43:59.647916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d129986-cd85-4486-a2ed-0279448f6638 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:43:59.651433 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:43:59.652069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:43:59.652354 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:43:59.652670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:43:59.661423 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:43:59.662853 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e8812-04bb-47e3-a183-70d4cae9bc50 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:43:59.672924 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 676/1346] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:43:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:03.933470 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afa331-b1c2-470c-9a31-fc8cbb04f3c9 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:03.934279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afa331-b1c2-470c-9a31-fc8cbb04f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:03.934279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afa331-b1c2-470c-9a31-fc8cbb04f3c9 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:03.935383 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 671/1347] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:03.943515 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c62-091c-4940-a45d-7bde3607cc71 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:03.944104 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c62-091c-4940-a45d-7bde3607cc71 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:03.944305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b82c62-091c-4940-a45d-7bde3607cc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:03.971220 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b3b82c62-091c-4940-a45d-7bde3607cc71 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:03.972447 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b3b82c62-091c-4940-a45d-7bde3607cc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:03.982236 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 677/1348] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:04.021911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afa331-b1c2-470c-9a31-fc8cbb04f3c9 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:04.033347 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afa331-b1c2-470c-9a31-fc8cbb04f3c9 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:04.038220 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:04.038946 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:04.039319 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:04.039669 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:04.054243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:04.056137 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe3391-c365-4b4f-9a2b-4886fafe6e17 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:04.070001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 672/1349] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:10.727263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec4e2-2834-4e9c-9301-dd038247e0f6 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:10.728714 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec4e2-2834-4e9c-9301-dd038247e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:10.729282 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ec4e2-2834-4e9c-9301-dd038247e0f6 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:10.733358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 678/1350] 10.222.0.21 () {54 vars in 921 bytes} [Wed May 25 13:44:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:10.743184 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ec7fc-a09d-4aa4-8c1e-a41eec390639 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:10.743707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ec7fc-a09d-4aa4-8c1e-a41eec390639 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:10.743849 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ec7fc-a09d-4aa4-8c1e-a41eec390639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:10.868593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-a74ec7fc-a09d-4aa4-8c1e-a41eec390639 None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:10.868593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-a74ec7fc-a09d-4aa4-8c1e-a41eec390639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:11.063146 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 673/1351] 10.222.0.21 () {58 vars in 1018 bytes} [Wed May 25 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:13.962577 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=13 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:16.105244 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ad92-917a-4608-84dc-4a2b52dfa01c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:16.107965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ad92-917a-4608-84dc-4a2b52dfa01c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:16.108949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ad92-917a-4608-84dc-4a2b52dfa01c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:16.113457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 679/1352] 10.222.0.21 () {54 vars in 921 bytes} [Wed May 25 13:44:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:16.119280 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eba0f-0da5-44e4-9da9-a1368905deea None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:16.119815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eba0f-0da5-44e4-9da9-a1368905deea None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:16.119901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5eba0f-0da5-44e4-9da9-a1368905deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:16.140670 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-7a5eba0f-0da5-44e4-9da9-a1368905deea None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:16.141922 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-7a5eba0f-0da5-44e4-9da9-a1368905deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:16.154416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 674/1353] 10.222.0.21 () {58 vars in 1018 bytes} [Wed May 25 13:44:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:18.626389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e34cd7-9652-4f16-8c90-5c1f51846697 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:18.627069 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e34cd7-9652-4f16-8c90-5c1f51846697 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:18.627620 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e34cd7-9652-4f16-8c90-5c1f51846697 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:18.630552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 680/1354] 10.222.0.21 () {54 vars in 921 bytes} [Wed May 25 13:44:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:18.636517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e460c4-d78a-4862-b820-e04870fe45c3 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:18.637106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e460c4-d78a-4862-b820-e04870fe45c3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:18.637713 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e460c4-d78a-4862-b820-e04870fe45c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:18.660155 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-74e460c4-d78a-4862-b820-e04870fe45c3 None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:18.661545 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-74e460c4-d78a-4862-b820-e04870fe45c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:18.670808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 675/1355] 10.222.0.21 () {58 vars in 1018 bytes} [Wed May 25 13:44:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:22.739183 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296969ce-94b7-4b0e-baf9-2bbb85d45d87 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:22.740328 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296969ce-94b7-4b0e-baf9-2bbb85d45d87 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:22.740855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296969ce-94b7-4b0e-baf9-2bbb85d45d87 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:22.742788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 681/1356] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:22.748889 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1e10-5853-4e8d-b180-290ff3e49b2f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:22.749460 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1e10-5853-4e8d-b180-290ff3e49b2f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:22.749670 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a1e10-5853-4e8d-b180-290ff3e49b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:22.771047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-1e5a1e10-5853-4e8d-b180-290ff3e49b2f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:22.772525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1e5a1e10-5853-4e8d-b180-290ff3e49b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:22.783224 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 676/1357] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:22.850983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490067-d944-403e-8166-306b64eea6d7 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:22.851284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490067-d944-403e-8166-306b64eea6d7 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:22.851647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490067-d944-403e-8166-306b64eea6d7 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:22.853443 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 682/1358] 10.222.0.21 () {54 vars in 961 bytes} [Wed May 25 13:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:22.864782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae051-7b4e-4724-9912-e2c0f7db19a3 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:22.865106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae051-7b4e-4724-9912-e2c0f7db19a3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:22.865534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ae051-7b4e-4724-9912-e2c0f7db19a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:22.923593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:22.936380 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-635ae051-7b4e-4724-9912-e2c0f7db19a3 None None] MFA Rules not processed for user `a21198d279bd499fbd1e2f42798f3918`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:22.937927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-635ae051-7b4e-4724-9912-e2c0f7db19a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:23.040985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 677/1359] 10.222.0.21 () {58 vars in 1058 bytes} [Wed May 25 13:44:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:23.051408 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490067-d944-403e-8166-306b64eea6d7 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:23.067881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490067-d944-403e-8166-306b64eea6d7 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:23.076168 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:23.077056 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:23.077416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:23.077811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:23.091795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:23.093565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2baaafe-4cb0-4365-be03-2b5e338055e7 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:23.105979 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 683/1360] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:25.817828 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db839b80-30c8-4f11-9078-f89ed3bf7a34 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:25.818565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db839b80-30c8-4f11-9078-f89ed3bf7a34 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:25.818565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db839b80-30c8-4f11-9078-f89ed3bf7a34 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:25.820162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 678/1361] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:25.826257 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c52c-5073-4f4c-8940-f84d783434e3 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:25.827243 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c52c-5073-4f4c-8940-f84d783434e3 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:25.827528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c52c-5073-4f4c-8940-f84d783434e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:25.851883 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-d426c52c-5073-4f4c-8940-f84d783434e3 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:25.853348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-d426c52c-5073-4f4c-8940-f84d783434e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:25.867752 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 684/1362] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:25.931143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db839b80-30c8-4f11-9078-f89ed3bf7a34 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:25.951386 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db839b80-30c8-4f11-9078-f89ed3bf7a34 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:25.964174 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:25.965177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:25.965466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:25.965902 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:25.976641 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:25.978357 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bedff-6a9c-4855-96e4-7ffab3df93f6 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:25.994738 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 679/1363] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:28.839367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c50ba-4a28-4a4a-bc0b-8cc205f25bdb None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.839894 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c50ba-4a28-4a4a-bc0b-8cc205f25bdb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.840032 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c50ba-4a28-4a4a-bc0b-8cc205f25bdb None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.841358 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 685/1364] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:28.847676 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687a5f-8530-4cdf-bb7a-c7abfda77a0d None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.848181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687a5f-8530-4cdf-bb7a-c7abfda77a0d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.848181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58687a5f-8530-4cdf-bb7a-c7abfda77a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.868881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-58687a5f-8530-4cdf-bb7a-c7abfda77a0d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:28.871154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-58687a5f-8530-4cdf-bb7a-c7abfda77a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:28.888433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 680/1365] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:28.944337 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64127e90-2645-42a0-84e1-f4adad738c29 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.944758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64127e90-2645-42a0-84e1-f4adad738c29 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.945146 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64127e90-2645-42a0-84e1-f4adad738c29 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.946711 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 686/1366] 10.222.0.21 () {54 vars in 961 bytes} [Wed May 25 13:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:28.954225 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc82bb-964c-4483-947a-e3223d98da9f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:28.954517 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc82bb-964c-4483-947a-e3223d98da9f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:28.954862 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc82bb-964c-4483-947a-e3223d98da9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:28.977846 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-95dc82bb-964c-4483-947a-e3223d98da9f None None] MFA Rules not processed for user `a21198d279bd499fbd1e2f42798f3918`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:28.979064 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-95dc82bb-964c-4483-947a-e3223d98da9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:28.989279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 681/1367] 10.222.0.21 () {58 vars in 1058 bytes} [Wed May 25 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:28.997817 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64127e90-2645-42a0-84e1-f4adad738c29 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:29.022852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64127e90-2645-42a0-84e1-f4adad738c29 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:29.031886 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:29.032874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:29.033014 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:29.033191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:29.044788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:29.046543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939bfca-710b-4a20-bf21-a9ccc5192c3d service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:29.058729 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 687/1368] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:31.710154 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1f43-6efb-44ec-8591-d9154861d588 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.710746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1f43-6efb-44ec-8591-d9154861d588 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.710746 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d1f43-6efb-44ec-8591-d9154861d588 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.712657 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 682/1369] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:31.720832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab74e-b59b-474b-be86-66afc38b2e09 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.721428 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab74e-b59b-474b-be86-66afc38b2e09 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.721845 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab74e-b59b-474b-be86-66afc38b2e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.755757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-766ab74e-b59b-474b-be86-66afc38b2e09 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:31.758524 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-766ab74e-b59b-474b-be86-66afc38b2e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:31.784593 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 688/1370] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:31.861349 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1f43-6efb-44ec-8591-d9154861d588 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:31.889446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d1f43-6efb-44ec-8591-d9154861d588 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:31.899034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:31.901001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:31.901957 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:31.902739 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:31.922330 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:31.928979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6003b6-762c-4d3b-bf88-2ccff478d053 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:31.956774 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 683/1371] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:32.120093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38f0b-8aaa-4f91-a75f-ca64a61b2f02 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:32.120326 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38f0b-8aaa-4f91-a75f-ca64a61b2f02 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:32.120500 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd38f0b-8aaa-4f91-a75f-ca64a61b2f02 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:32.122315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 689/1372] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:32.128985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9a61-2cd4-48ef-aefe-696670680f0f None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:32.129294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9a61-2cd4-48ef-aefe-696670680f0f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:32.129885 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a9a61-2cd4-48ef-aefe-696670680f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:32.152202 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-db7a9a61-2cd4-48ef-aefe-696670680f0f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:32.153653 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-db7a9a61-2cd4-48ef-aefe-696670680f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:32.164609 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 684/1373] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:32.467922 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e9df-e18f-44d1-9766-7ffdeeb651e9 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:32.468255 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e9df-e18f-44d1-9766-7ffdeeb651e9 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:32.468542 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011e9df-e18f-44d1-9766-7ffdeeb651e9 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:32.470642 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 690/1374] 10.222.0.21 () {54 vars in 963 bytes} [Wed May 25 13:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:32.478521 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db91af-b702-4ef7-876a-5bb8cd49a729 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:32.478901 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db91af-b702-4ef7-876a-5bb8cd49a729 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:32.479193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db91af-b702-4ef7-876a-5bb8cd49a729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:32.546921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-58db91af-b702-4ef7-876a-5bb8cd49a729 None None] MFA Rules not processed for user `afc0939b6c0e4e6b81bc8337065cdc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:32.548626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-58db91af-b702-4ef7-876a-5bb8cd49a729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:32.636147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 685/1375] 10.222.0.21 () {58 vars in 1060 bytes} [Wed May 25 13:44:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:32.645670 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e9df-e18f-44d1-9766-7ffdeeb651e9 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:32.660289 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011e9df-e18f-44d1-9766-7ffdeeb651e9 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:32.664263 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc0939b6c0e4e6b81bc8337065cdc5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:32.665028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:32.665348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:32.665639 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:32.677982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:32.679545 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651d4db-dc07-420f-9e67-72a1e4e32788 service cinder] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:32.691540 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 691/1376] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:34.614711 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5b71-5d42-45a2-a88a-11ff2317a9d0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:34.615346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5b71-5d42-45a2-a88a-11ff2317a9d0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:34.615346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff5b71-5d42-45a2-a88a-11ff2317a9d0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:34.616780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 686/1377] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:34.624123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f459a8-679b-4be4-b054-810603b32107 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:34.624935 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f459a8-679b-4be4-b054-810603b32107 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:34.625018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f459a8-679b-4be4-b054-810603b32107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:34.651157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-84f459a8-679b-4be4-b054-810603b32107 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:34.651405 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-84f459a8-679b-4be4-b054-810603b32107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:34.660114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 692/1378] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:34.710004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff5b71-5d42-45a2-a88a-11ff2317a9d0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:34.726289 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff5b71-5d42-45a2-a88a-11ff2317a9d0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:34.732872 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:34.733722 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:34.734009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:34.734287 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:34.745483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:34.747313 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de7d74-b077-4066-8079-a517851ca7aa service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:34.758419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:34.761265 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 687/1379] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:37.298647 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d352-937a-4b91-b935-543760a5eb3d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:37.299559 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d352-937a-4b91-b935-543760a5eb3d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:37.299852 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9d352-937a-4b91-b935-543760a5eb3d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:37.301345 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 693/1380] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:37.307526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c1157-eccf-475a-92c2-9c48e1e67a30 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:37.308071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c1157-eccf-475a-92c2-9c48e1e67a30 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:37.308071 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c1157-eccf-475a-92c2-9c48e1e67a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:37.328244 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-349c1157-eccf-475a-92c2-9c48e1e67a30 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:37.330172 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-349c1157-eccf-475a-92c2-9c48e1e67a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:37.340275 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 688/1381] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:37.388688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9d352-937a-4b91-b935-543760a5eb3d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:37.405740 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9d352-937a-4b91-b935-543760a5eb3d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:37.413185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:37.414217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:37.414217 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:37.414422 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:37.423990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:37.425673 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8221bf-5d11-4a71-a2e0-f0875a3d260f service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:37.437154 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 694/1382] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:39.777169 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b540c8-dc42-4948-be91-fe3eb82c4ae1 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:39.777705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b540c8-dc42-4948-be91-fe3eb82c4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:39.777705 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b540c8-dc42-4948-be91-fe3eb82c4ae1 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:39.779023 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 689/1383] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:39.784505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f2208-cdb5-4279-855f-3e5b4d8d663a None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:39.784988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f2208-cdb5-4279-855f-3e5b4d8d663a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:39.785120 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f2208-cdb5-4279-855f-3e5b4d8d663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:39.803303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-bd4f2208-cdb5-4279-855f-3e5b4d8d663a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:39.804492 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-bd4f2208-cdb5-4279-855f-3e5b4d8d663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:39.812525 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 695/1384] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:39.855110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b540c8-dc42-4948-be91-fe3eb82c4ae1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:39.871080 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b540c8-dc42-4948-be91-fe3eb82c4ae1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:39.878437 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:39.879079 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:39.879420 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:39.879643 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:39.890606 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:39.893218 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43620fdf-79ab-4e9d-b662-5acdfc365cae service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:39.905742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 690/1385] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:42.215403 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974f69-b9d5-4e96-bd70-193229c33a73 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:42.216528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974f69-b9d5-4e96-bd70-193229c33a73 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:42.216528 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64974f69-b9d5-4e96-bd70-193229c33a73 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:42.218305 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 696/1386] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:42.223974 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4cf00-23ea-449a-b073-8df5cfcec8c0 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:42.224498 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4cf00-23ea-449a-b073-8df5cfcec8c0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:42.224620 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4cf00-23ea-449a-b073-8df5cfcec8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:42.242927 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-e0b4cf00-23ea-449a-b073-8df5cfcec8c0 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:42.244151 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e0b4cf00-23ea-449a-b073-8df5cfcec8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:42.252936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 691/1387] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:42.296841 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974f69-b9d5-4e96-bd70-193229c33a73 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:42.314643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64974f69-b9d5-4e96-bd70-193229c33a73 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:42.320325 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:42.321004 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:42.321296 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:42.321570 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:42.330665 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:42.332260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09917ab-75f2-44f9-ab18-c841c1e8db76 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:42.342316 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 697/1388] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:44.615294 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471ae1-3c95-46c9-87c7-fda8231029c1 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:44.615986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471ae1-3c95-46c9-87c7-fda8231029c1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:44.615986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6471ae1-3c95-46c9-87c7-fda8231029c1 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:44.617264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 692/1389] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:44.622633 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79053-dc72-4da8-ac90-d6847fe03ba1 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:44.623096 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79053-dc72-4da8-ac90-d6847fe03ba1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:44.623199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b79053-dc72-4da8-ac90-d6847fe03ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:44.629965 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=43 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:44.644726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b0b79053-dc72-4da8-ac90-d6847fe03ba1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:44.646404 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b0b79053-dc72-4da8-ac90-d6847fe03ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:44.657035 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 698/1390] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:44.712180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6471ae1-3c95-46c9-87c7-fda8231029c1 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:44.731587 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6471ae1-3c95-46c9-87c7-fda8231029c1 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:44.739393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:44.740303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:44.740704 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:44.741333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:44.754324 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:44.756593 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae2607-7e39-4fd4-93bd-913f7250c14a service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:44.769002 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 693/1391] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:47.600322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b58a90-cd21-4140-9830-3bc107cfef3a None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:47.600808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b58a90-cd21-4140-9830-3bc107cfef3a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:47.600913 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b58a90-cd21-4140-9830-3bc107cfef3a None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:47.602277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 699/1392] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:47.607597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e20b9f-0640-46e9-b0f3-0e16a0f0887c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:47.608066 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e20b9f-0640-46e9-b0f3-0e16a0f0887c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:47.608188 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e20b9f-0640-46e9-b0f3-0e16a0f0887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:47.627549 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-66e20b9f-0640-46e9-b0f3-0e16a0f0887c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:47.628787 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-66e20b9f-0640-46e9-b0f3-0e16a0f0887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:47.637018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 694/1393] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:47.690619 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b58a90-cd21-4140-9830-3bc107cfef3a None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:47.705597 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b58a90-cd21-4140-9830-3bc107cfef3a None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:47.711166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:47.711821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:47.712111 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:47.712396 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:47.721993 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:47.723607 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a041d2-1c2c-4bf1-975a-cc3fa1554d18 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:47.733603 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 700/1394] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:49.520193 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41111f44-3a0d-4ec4-b5f7-f35e92b497dc None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:49.520932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41111f44-3a0d-4ec4-b5f7-f35e92b497dc None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:49.520932 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41111f44-3a0d-4ec4-b5f7-f35e92b497dc None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:49.522181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 695/1395] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:49.527706 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ecac-6961-4c9a-9c51-305c53a76e80 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:49.528165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ecac-6961-4c9a-9c51-305c53a76e80 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:49.528431 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ecac-6961-4c9a-9c51-305c53a76e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:49.548412 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-74c0ecac-6961-4c9a-9c51-305c53a76e80 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:49.549685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-74c0ecac-6961-4c9a-9c51-305c53a76e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:49.558179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 701/1396] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:49.616539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870361f-57e0-46d5-a431-9baaf7d9f035 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:49.616875 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870361f-57e0-46d5-a431-9baaf7d9f035 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:49.617158 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870361f-57e0-46d5-a431-9baaf7d9f035 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:49.618636 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 696/1397] 10.222.0.21 () {54 vars in 962 bytes} [Wed May 25 13:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:49.624335 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283720-4091-428f-a311-896044ad0683 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:49.624666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283720-4091-428f-a311-896044ad0683 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:49.624988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283720-4091-428f-a311-896044ad0683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:49.682516 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-19283720-4091-428f-a311-896044ad0683 None None] MFA Rules not processed for user `1f45bec6796b4e10a1b2a598e765fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:49.685395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-19283720-4091-428f-a311-896044ad0683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:49.756526 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 702/1398] 10.222.0.21 () {58 vars in 1059 bytes} [Wed May 25 13:44:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:49.764432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870361f-57e0-46d5-a431-9baaf7d9f035 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:49.776147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870361f-57e0-46d5-a431-9baaf7d9f035 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:49.779602 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f45bec6796b4e10a1b2a598e765fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:49.780243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:49.780525 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:49.780831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:49.790921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:49.792370 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf5163c-994b-43a3-8998-cca7b12aafb9 service glance] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:49.802157 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 697/1399] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:50.019258 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b980-bc7d-4573-9e1e-4f4588987f82 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.019556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b980-bc7d-4573-9e1e-4f4588987f82 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.019999 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b980-bc7d-4573-9e1e-4f4588987f82 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.021268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 703/1400] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:44:50.026911 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf884d-e91a-4485-b0bc-a1a3d504f8b1 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.027143 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf884d-e91a-4485-b0bc-a1a3d504f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.027479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf884d-e91a-4485-b0bc-a1a3d504f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.047567 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-30bf884d-e91a-4485-b0bc-a1a3d504f8b1 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:50.049585 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-30bf884d-e91a-4485-b0bc-a1a3d504f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:50.057406 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 698/1401] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:44:50.103200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19b980-bc7d-4573-9e1e-4f4588987f82 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:50.118409 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19b980-bc7d-4573-9e1e-4f4588987f82 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:50.123956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:50.124632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.124917 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.125224 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.133990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:50.135616 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f859f9-4be7-4af9-8a0f-0f66ff69535f service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:50.145389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 704/1402] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:44:50.373364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c23-f0f7-4eac-8799-101a7106ac9a None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.373988 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c23-f0f7-4eac-8799-101a7106ac9a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.374443 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a5c23-f0f7-4eac-8799-101a7106ac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.434089 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-ca1a5c23-f0f7-4eac-8799-101a7106ac9a None None] MFA Rules not processed for user `907ba93949a24ceeb776c213a75daac6`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:50.435295 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-ca1a5c23-f0f7-4eac-8799-101a7106ac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:50.500353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 699/1403] 10.222.0.21 () {58 vars in 1009 bytes} [Wed May 25 13:44:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:50.541942 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8223-a188-424d-a42c-98451f2a9380 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.542482 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8223-a188-424d-a42c-98451f2a9380 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.542482 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8223-a188-424d-a42c-98451f2a9380 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.543880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 705/1404] 10.222.0.21 () {54 vars in 942 bytes} [Wed May 25 13:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:50.550507 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46b35-ead8-41d4-8ea6-9a85513b63a8 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.550967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46b35-ead8-41d4-8ea6-9a85513b63a8 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.551194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46b35-ead8-41d4-8ea6-9a85513b63a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.602626 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-a8c46b35-ead8-41d4-8ea6-9a85513b63a8 None None] MFA Rules not processed for user `17fa4d0ea2da404c91576db368cf28bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:50.603871 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-a8c46b35-ead8-41d4-8ea6-9a85513b63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:50.638783 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=100 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:44:50.679125 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 700/1405] 10.222.0.21 () {58 vars in 1039 bytes} [Wed May 25 13:44:50 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:50.689872 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8223-a188-424d-a42c-98451f2a9380 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:50.704992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8223-a188-424d-a42c-98451f2a9380 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:50.710055 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17fa4d0ea2da404c91576db368cf28bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:50.710711 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:50.711013 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:50.711272 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:50.720312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:50.721939 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4ea0b-8c2b-4915-bed7-9a7025b51f87 service swift] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:50.730140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 706/1406] 10.222.0.21 () {60 vars in 1384 bytes} [Wed May 25 13:44:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 13:44:52.438386 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85004-3cdd-4dc6-af00-6462caf41ab0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:52.438980 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85004-3cdd-4dc6-af00-6462caf41ab0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:52.439092 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85004-3cdd-4dc6-af00-6462caf41ab0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:52.440660 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 701/1407] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:44:52.445956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f71e9-6834-4b53-b928-9735aa61151e None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:52.446748 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f71e9-6834-4b53-b928-9735aa61151e None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:52.447188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f71e9-6834-4b53-b928-9735aa61151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:52.468420 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-711f71e9-6834-4b53-b928-9735aa61151e None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:44:52.470565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-711f71e9-6834-4b53-b928-9735aa61151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:44:52.479499 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 707/1408] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:44:52.527843 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85004-3cdd-4dc6-af00-6462caf41ab0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:44:52.543905 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85004-3cdd-4dc6-af00-6462caf41ab0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:44:52.550397 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:44:52.551504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:44:52.551855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:44:52.552214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:44:52.562728 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:44:52.564540 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0cdc9-f95a-4259-ab83-1c83b179455a service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:44:52.575419 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 702/1409] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:02.462742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=64 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:02.570700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=21 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:15.538089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94168d8-0913-4b2b-8ff7-9f106370c7e2 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:15.538743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94168d8-0913-4b2b-8ff7-9f106370c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:15.538743 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94168d8-0913-4b2b-8ff7-9f106370c7e2 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:15.540054 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 708/1410] 10.222.0.21 () {54 vars in 918 bytes} [Wed May 25 13:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:15.546037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5f71-8bbb-4b9c-a18a-c176a7320be4 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:15.546526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5f71-8bbb-4b9c-a18a-c176a7320be4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:15.546640 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5f71-8bbb-4b9c-a18a-c176a7320be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:15.565546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-711f5f71-8bbb-4b9c-a18a-c176a7320be4 None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:15.566757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-711f5f71-8bbb-4b9c-a18a-c176a7320be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:15.575454 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 703/1411] 10.222.0.21 () {58 vars in 1015 bytes} [Wed May 25 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:15.590179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614a98-27a4-4999-aff1-d7ba6d6eb982 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:15.590473 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614a98-27a4-4999-aff1-d7ba6d6eb982 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:15.590757 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33614a98-27a4-4999-aff1-d7ba6d6eb982 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:15.592921 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 709/1412] 10.222.0.21 () {54 vars in 918 bytes} [Wed May 25 13:45:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:15.597935 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf3ef-a99e-4069-aa4d-77e9d9f516cc None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:15.598234 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf3ef-a99e-4069-aa4d-77e9d9f516cc None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:15.598518 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf3ef-a99e-4069-aa4d-77e9d9f516cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:15.618168 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-793cf3ef-a99e-4069-aa4d-77e9d9f516cc None None] MFA Rules not processed for user `a21198d279bd499fbd1e2f42798f3918`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:15.619372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-793cf3ef-a99e-4069-aa4d-77e9d9f516cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:15.628784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 704/1413] 10.222.0.21 () {58 vars in 1015 bytes} [Wed May 25 13:45:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:18.352252 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627a27f-76c7-4c82-9cce-5d0973a89484 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:18.352803 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627a27f-76c7-4c82-9cce-5d0973a89484 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:18.352936 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627a27f-76c7-4c82-9cce-5d0973a89484 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:18.354205 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:18.360333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9caf-a625-4b04-8a22-8b7b141ff186 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:18.360827 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9caf-a625-4b04-8a22-8b7b141ff186 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:18.360959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9caf-a625-4b04-8a22-8b7b141ff186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:18.380147 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-803d9caf-a625-4b04-8a22-8b7b141ff186 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:18.381374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-803d9caf-a625-4b04-8a22-8b7b141ff186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:18.390050 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:18.433521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627a27f-76c7-4c82-9cce-5d0973a89484 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:18.448424 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627a27f-76c7-4c82-9cce-5d0973a89484 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:18.456365 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:18.457047 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:18.457330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:18.457601 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:18.467089 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:18.468747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d4be2-408f-4165-8d07-4d53e8be663c service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:18.478304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 711/1416] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:19.491996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef43d2b-be99-49f2-932e-fa1fd853df7f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.492546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef43d2b-be99-49f2-932e-fa1fd853df7f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.492546 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef43d2b-be99-49f2-932e-fa1fd853df7f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.493975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:19.499687 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bccfb-06fe-4db2-b7ca-81426febd7c2 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.500167 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bccfb-06fe-4db2-b7ca-81426febd7c2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.500276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bccfb-06fe-4db2-b7ca-81426febd7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.519777 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-137bccfb-06fe-4db2-b7ca-81426febd7c2 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:19.521019 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-137bccfb-06fe-4db2-b7ca-81426febd7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:19.529100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:19.737510 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef43d2b-be99-49f2-932e-fa1fd853df7f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:19.753038 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef43d2b-be99-49f2-932e-fa1fd853df7f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:19.758519 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:19.759163 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:19.759444 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:19.759715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:19.768793 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:19.770394 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96de397-270a-48ee-84ea-0d2731a8d0d1 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:19.780341 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 707/1419] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:20.738769 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90175f9-4af3-4b4e-b93b-2ecced7e9410 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:20.739336 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90175f9-4af3-4b4e-b93b-2ecced7e9410 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:20.739452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90175f9-4af3-4b4e-b93b-2ecced7e9410 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:20.741029 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:20.745952 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043960f-2e21-4bbc-a7ba-e292588d9581 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:20.746392 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043960f-2e21-4bbc-a7ba-e292588d9581 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:20.746514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043960f-2e21-4bbc-a7ba-e292588d9581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:20.765206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5043960f-2e21-4bbc-a7ba-e292588d9581 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:20.766415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5043960f-2e21-4bbc-a7ba-e292588d9581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:20.774268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:20.951780 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90175f9-4af3-4b4e-b93b-2ecced7e9410 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:20.972389 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90175f9-4af3-4b4e-b93b-2ecced7e9410 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:20.980432 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:20.981156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:20.981444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:20.981722 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:20.990750 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:20.992359 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90ebca-c2a7-4cfd-813a-4bb88ab95989 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:21.003223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 714/1422] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:21.968743 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e894-97c5-4173-89bd-fa5cc0b302c3 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:21.969339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e894-97c5-4173-89bd-fa5cc0b302c3 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:21.969339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e894-97c5-4173-89bd-fa5cc0b302c3 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:21.970697 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:45:21.976610 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911715-d2a0-489b-b93a-22a5211e830b None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:21.977062 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911715-d2a0-489b-b93a-22a5211e830b None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:21.977265 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60911715-d2a0-489b-b93a-22a5211e830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:21.996169 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-60911715-d2a0-489b-b93a-22a5211e830b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:21.998011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-60911715-d2a0-489b-b93a-22a5211e830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:22.006452 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:22.158655 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530e894-97c5-4173-89bd-fa5cc0b302c3 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:22.173651 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530e894-97c5-4173-89bd-fa5cc0b302c3 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:22.179506 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:22.180158 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:22.180447 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:22.180757 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:22.190479 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:22.192121 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b4781-df9f-4c93-b1b6-e7a63412a5f8 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:22.214242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 710/1425] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:23.146041 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25dcf-b3fa-42aa-b38d-5fb368201988 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:23.146646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25dcf-b3fa-42aa-b38d-5fb368201988 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:23.146646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25dcf-b3fa-42aa-b38d-5fb368201988 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:23.148486 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:23.154260 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877a7e-5289-4e8a-aff8-231e3e1a6af8 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:23.154810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877a7e-5289-4e8a-aff8-231e3e1a6af8 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:23.154810 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c877a7e-5289-4e8a-aff8-231e3e1a6af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:23.173597 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-4c877a7e-5289-4e8a-aff8-231e3e1a6af8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:23.175110 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-4c877a7e-5289-4e8a-aff8-231e3e1a6af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:23.182904 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:23.336821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25dcf-b3fa-42aa-b38d-5fb368201988 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:23.351352 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25dcf-b3fa-42aa-b38d-5fb368201988 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:23.356859 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:23.357527 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:23.357812 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:23.358092 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:23.366821 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:23.368799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd677ad-a038-4011-abe3-fa8686c3a864 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:23.381225 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 717/1428] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:24.274778 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edcae-ef93-4ef3-b806-a8324555a3b5 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:24.275350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edcae-ef93-4ef3-b806-a8324555a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:24.275350 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2edcae-ef93-4ef3-b806-a8324555a3b5 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:24.276777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:24.282742 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d3da-21da-46dd-bedf-67e8a9a87588 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:24.283212 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d3da-21da-46dd-bedf-67e8a9a87588 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:24.283306 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d3da-21da-46dd-bedf-67e8a9a87588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:24.304635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0d37d3da-21da-46dd-bedf-67e8a9a87588 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:24.305855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0d37d3da-21da-46dd-bedf-67e8a9a87588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:24.313983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:24.466724 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2edcae-ef93-4ef3-b806-a8324555a3b5 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:24.483520 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2edcae-ef93-4ef3-b806-a8324555a3b5 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:24.489055 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:24.489700 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:24.489989 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:24.490268 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:24.500095 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:24.501679 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6ba90-4623-4b85-9a3a-83d7c8f65780 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:24.512725 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 713/1431] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:25.382707 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f42b3-6886-4284-b3dc-71eee532efce None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:25.383276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f42b3-6886-4284-b3dc-71eee532efce None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:25.383276 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f42b3-6886-4284-b3dc-71eee532efce None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:25.384700 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:25.390612 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca3a3-fc50-41a7-bd45-05e95691142e None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:25.391096 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca3a3-fc50-41a7-bd45-05e95691142e None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:25.391212 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ca3a3-fc50-41a7-bd45-05e95691142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:25.410863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-fa1ca3a3-fc50-41a7-bd45-05e95691142e None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:25.412084 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-fa1ca3a3-fc50-41a7-bd45-05e95691142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:25.420766 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:25.576711 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f42b3-6886-4284-b3dc-71eee532efce None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:25.592017 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f42b3-6886-4284-b3dc-71eee532efce None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:25.597635 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:25.598303 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:25.598592 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:25.598874 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:25.608658 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:25.610302 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b07d9c-6788-484f-a13a-24000eb79b32 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:25.620955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 720/1434] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:26.505239 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b6bc2-c32e-42f0-82b7-085b0a444961 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:26.505895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b6bc2-c32e-42f0-82b7-085b0a444961 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:26.505895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b6bc2-c32e-42f0-82b7-085b0a444961 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:26.507227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:26.512623 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318982-d8f3-42de-8282-dd6d0948437a None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:26.513116 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318982-d8f3-42de-8282-dd6d0948437a None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:26.513222 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e318982-d8f3-42de-8282-dd6d0948437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:26.532026 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-3e318982-d8f3-42de-8282-dd6d0948437a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:26.533284 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-3e318982-d8f3-42de-8282-dd6d0948437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:26.541298 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:26.696586 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b6bc2-c32e-42f0-82b7-085b0a444961 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:26.711623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b6bc2-c32e-42f0-82b7-085b0a444961 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:26.717616 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:26.718249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:26.718538 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:26.718812 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:26.728243 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:26.729942 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b2afe-9b20-4593-9171-8211e4ce9b98 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:26.740552 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 716/1437] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:27.624166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabeed-5fbc-4e32-8df5-cb17d3fd6f1f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:27.624739 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabeed-5fbc-4e32-8df5-cb17d3fd6f1f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:27.624858 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfabeed-5fbc-4e32-8df5-cb17d3fd6f1f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:27.626141 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:27.631369 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde577-61fc-4008-a00b-b7338e1732eb None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:27.631831 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde577-61fc-4008-a00b-b7338e1732eb None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:27.631949 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cde577-61fc-4008-a00b-b7338e1732eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:27.649805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-76cde577-61fc-4008-a00b-b7338e1732eb None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:27.651004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-76cde577-61fc-4008-a00b-b7338e1732eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:27.659374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:27.809974 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabeed-5fbc-4e32-8df5-cb17d3fd6f1f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:27.824981 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfabeed-5fbc-4e32-8df5-cb17d3fd6f1f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:27.831039 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:27.831786 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:27.832119 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:27.832444 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:27.841798 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:27.843481 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4884d-06ab-4720-b66b-517fef9522aa service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:27.853343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 723/1440] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:30.016948 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f444-71b0-447b-9486-c86e12918ba0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:30.017514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f444-71b0-447b-9486-c86e12918ba0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:30.017514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3995f444-71b0-447b-9486-c86e12918ba0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:30.019033 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:30.024904 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe490d-b089-45aa-a676-c628c2774bf8 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:30.025397 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe490d-b089-45aa-a676-c628c2774bf8 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:30.025493 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe490d-b089-45aa-a676-c628c2774bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:30.045688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-09fe490d-b089-45aa-a676-c628c2774bf8 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:30.046895 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-09fe490d-b089-45aa-a676-c628c2774bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:30.055213 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:45:30.094333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f444-71b0-447b-9486-c86e12918ba0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:30.106281 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3995f444-71b0-447b-9486-c86e12918ba0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:30.110203 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:30.110919 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:30.111291 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:30.111628 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:30.121811 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:30.123311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229c8cd-a345-4fd8-978e-48192c21407d service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:30.133897 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 719/1443] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:32.537510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245227-e274-4d8d-b61c-21a42359240f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:32.538066 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245227-e274-4d8d-b61c-21a42359240f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:32.538166 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245227-e274-4d8d-b61c-21a42359240f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:32.539489 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:32.545680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6fe1-4857-4ba1-b933-146c3170df34 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:32.546168 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6fe1-4857-4ba1-b933-146c3170df34 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:32.546264 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd6fe1-4857-4ba1-b933-146c3170df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:32.564907 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-dbdd6fe1-4857-4ba1-b933-146c3170df34 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:32.566117 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-dbdd6fe1-4857-4ba1-b933-146c3170df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:32.574197 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:32.614074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10245227-e274-4d8d-b61c-21a42359240f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:32.630761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10245227-e274-4d8d-b61c-21a42359240f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:32.634691 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:32.635343 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:32.635622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:32.635916 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:32.646256 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:32.647685 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2f8788-6593-4178-a0ae-01a1e55f4f74 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:32.657223 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 726/1446] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:35.248720 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:35.249333 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:35.249706 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:35.252139 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:35.264117 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5d7df-f779-4d62-958f-8b7c3b69a404 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:35.264703 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5d7df-f779-4d62-958f-8b7c3b69a404 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:35.264808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5d7df-f779-4d62-958f-8b7c3b69a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:35.282461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-45c5d7df-f779-4d62-958f-8b7c3b69a404 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:35.283672 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-45c5d7df-f779-4d62-958f-8b7c3b69a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:35.291478 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:35.331177 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:35.333130 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:35.346563 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349ab54-786e-41ec-8d05-5bc889bdbf06 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:35.350246 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:35.350893 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:35.351181 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:35.351459 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:35.361745 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:35.363199 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07042961-00aa-45c4-98c3-7354cb917ea8 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:35.374537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 722/1449] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:37.764596 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f1c5-5a1c-4241-b625-ebe35b4eded1 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:37.765185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f1c5-5a1c-4241-b625-ebe35b4eded1 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:37.765185 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f1c5-5a1c-4241-b625-ebe35b4eded1 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:37.766512 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:37.771975 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ecc14-764f-4ede-9e6c-c3f34d8b0bd9 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:37.772415 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ecc14-764f-4ede-9e6c-c3f34d8b0bd9 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:37.772548 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ecc14-764f-4ede-9e6c-c3f34d8b0bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:37.791112 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5f8ecc14-764f-4ede-9e6c-c3f34d8b0bd9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:37.793034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5f8ecc14-764f-4ede-9e6c-c3f34d8b0bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:37.801166 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:37.840138 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f1c5-5a1c-4241-b625-ebe35b4eded1 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:37.851028 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f1c5-5a1c-4241-b625-ebe35b4eded1 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:37.855156 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:37.855813 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:37.856103 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:37.856411 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:37.865973 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:37.867415 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c480ef-3cb8-4fd5-ba8b-eccf36328c44 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:37.878052 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 729/1452] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:45:40.242106 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0519-2ec1-4970-a3a3-f11197de2a34 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:40.242648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0519-2ec1-4970-a3a3-f11197de2a34 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:40.242648 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e0519-2ec1-4970-a3a3-f11197de2a34 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:40.243973 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:40.249943 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118780ff-6348-4f72-9158-893456481036 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:40.250395 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118780ff-6348-4f72-9158-893456481036 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:40.250495 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118780ff-6348-4f72-9158-893456481036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:40.269402 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-118780ff-6348-4f72-9158-893456481036 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:40.270591 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-118780ff-6348-4f72-9158-893456481036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:40.279123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:40.317554 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e0519-2ec1-4970-a3a3-f11197de2a34 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:40.329855 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e0519-2ec1-4970-a3a3-f11197de2a34 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:40.333367 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:40.334001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:40.334317 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:40.334594 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:40.344343 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:40.345821 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7d7d5-e4ef-4646-9347-3a3f0de68040 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:40.355399 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 725/1455] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:42.686940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511fe8-2ff4-4748-a368-479bc5e5f69f None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:42.687508 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511fe8-2ff4-4748-a368-479bc5e5f69f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:42.687625 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42511fe8-2ff4-4748-a368-479bc5e5f69f None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:42.689259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:42.694800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62cad4-1e1c-42ad-bc82-ac583846badd None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:42.695249 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62cad4-1e1c-42ad-bc82-ac583846badd None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:42.695359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62cad4-1e1c-42ad-bc82-ac583846badd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:42.701866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=70 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:42.715440 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-1c62cad4-1e1c-42ad-bc82-ac583846badd None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:42.716791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-1c62cad4-1e1c-42ad-bc82-ac583846badd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:42.724689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:42.765269 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42511fe8-2ff4-4748-a368-479bc5e5f69f None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:42.770682 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=64 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:45:42.778425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42511fe8-2ff4-4748-a368-479bc5e5f69f None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:42.782011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:42.782664 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:42.782951 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:42.783230 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:42.794624 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:42.796074 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b924047-7db8-4285-ab69-5e67503ecc36 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:42.806210 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 732/1458] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:45.107622 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a72d-c31e-40a8-897b-30147d766215 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.108211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a72d-c31e-40a8-897b-30147d766215 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.108211 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7a72d-c31e-40a8-897b-30147d766215 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.109573 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:45.114897 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08abdc-b53e-4fd8-ab17-8c6ad2470862 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.115350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08abdc-b53e-4fd8-ab17-8c6ad2470862 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.115455 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08abdc-b53e-4fd8-ab17-8c6ad2470862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.132923 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-db08abdc-b53e-4fd8-ab17-8c6ad2470862 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:45.134123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-db08abdc-b53e-4fd8-ab17-8c6ad2470862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:45.142018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:45.179916 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7a72d-c31e-40a8-897b-30147d766215 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:45.190886 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7a72d-c31e-40a8-897b-30147d766215 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:45.194465 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:45.195108 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:45.195389 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:45.195663 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:45.204986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:45.206433 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bb249-252c-4d5e-aeca-32f5ae1c1ee5 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:45.216335 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 728/1461] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:47.519479 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383be32-1aca-41ec-8f1d-66f988c5a604 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.520170 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383be32-1aca-41ec-8f1d-66f988c5a604 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.520170 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383be32-1aca-41ec-8f1d-66f988c5a604 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.521442 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:47.526780 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e333-90ef-4417-9aa6-d694356d8ffc None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.527198 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e333-90ef-4417-9aa6-d694356d8ffc None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.527353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807e333-90ef-4417-9aa6-d694356d8ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.545623 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5807e333-90ef-4417-9aa6-d694356d8ffc None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:47.547068 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5807e333-90ef-4417-9aa6-d694356d8ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:47.554887 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:47.592671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383be32-1aca-41ec-8f1d-66f988c5a604 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:47.604505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383be32-1aca-41ec-8f1d-66f988c5a604 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:47.609930 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:47.610584 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:47.610880 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:47.611161 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:47.621565 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:47.622988 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89e594-2196-489f-89a1-5700b484d191 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:47.632301 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 735/1464] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:49.872610 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4943f7-bf01-48b0-ac94-3cf123297572 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:49.873132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4943f7-bf01-48b0-ac94-3cf123297572 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:49.873132 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4943f7-bf01-48b0-ac94-3cf123297572 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:49.874467 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:45:49.880157 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25393e14-6a61-4f05-9712-8e4b28b552d9 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:49.880808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25393e14-6a61-4f05-9712-8e4b28b552d9 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:49.880808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25393e14-6a61-4f05-9712-8e4b28b552d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:49.902011 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-25393e14-6a61-4f05-9712-8e4b28b552d9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:49.903277 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-25393e14-6a61-4f05-9712-8e4b28b552d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:49.912143 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:49.952847 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4943f7-bf01-48b0-ac94-3cf123297572 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:49.965022 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4943f7-bf01-48b0-ac94-3cf123297572 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:49.968908 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:49.969565 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:49.969851 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:49.970131 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:49.982182 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:49.984951 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98909124-8a5d-4c58-9ece-9a2d1735d512 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:50.003134 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 731/1467] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:45:52.331772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24667781-9f56-4879-9207-87307cc0f9eb None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.333093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24667781-9f56-4879-9207-87307cc0f9eb None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.333093 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24667781-9f56-4879-9207-87307cc0f9eb None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.333955 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:45:52.342685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefaffa-d625-482c-aedf-baf771377d5b None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.344550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefaffa-d625-482c-aedf-baf771377d5b None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.344859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefaffa-d625-482c-aedf-baf771377d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.362993 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-6cefaffa-d625-482c-aedf-baf771377d5b None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:45:52.364282 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-6cefaffa-d625-482c-aedf-baf771377d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:45:52.375214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:45:52.411882 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24667781-9f56-4879-9207-87307cc0f9eb None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:45:52.422735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24667781-9f56-4879-9207-87307cc0f9eb None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:45:52.426304 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:45:52.426946 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:45:52.427232 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:45:52.427543 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:45:52.437770 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:45:52.439203 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1ca30-dd98-4799-be18-1a7c5e4933d4 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:45:52.449425 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 738/1470] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:02.357815 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=20 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:02.444457 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=23 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:26.986067 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b952d-b920-468b-b7dc-cc155c418827 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:26.986797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b952d-b920-468b-b7dc-cc155c418827 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:26.986797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b952d-b920-468b-b7dc-cc155c418827 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:26.988012 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 733/1471] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:26.994371 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84572-4c83-4533-83e5-945cb0558627 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:26.994883 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84572-4c83-4533-83e5-945cb0558627 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:26.994992 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d84572-4c83-4533-83e5-945cb0558627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:27.013246 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-97d84572-4c83-4533-83e5-945cb0558627 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:27.014451 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-97d84572-4c83-4533-83e5-945cb0558627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:27.022399 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 739/1472] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:46:27.070685 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a031-7710-49c8-bd0c-d4fce3b26af0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:27.070978 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a031-7710-49c8-bd0c-d4fce3b26af0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:27.071257 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70a031-7710-49c8-bd0c-d4fce3b26af0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:27.072666 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 734/1473] 10.222.0.21 () {54 vars in 962 bytes} [Wed May 25 13:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:27.077652 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05c24b-6aa1-490b-a505-71eb60dfb8bf None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:27.077960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05c24b-6aa1-490b-a505-71eb60dfb8bf None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:27.078237 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05c24b-6aa1-490b-a505-71eb60dfb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:27.097809 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-ec05c24b-6aa1-490b-a505-71eb60dfb8bf None None] MFA Rules not processed for user `1f45bec6796b4e10a1b2a598e765fcce`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:27.099186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-ec05c24b-6aa1-490b-a505-71eb60dfb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:27.106177 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 740/1474] 10.222.0.21 () {58 vars in 1059 bytes} [Wed May 25 13:46:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:27.113879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70a031-7710-49c8-bd0c-d4fce3b26af0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:27.126391 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70a031-7710-49c8-bd0c-d4fce3b26af0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:27.130709 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f45bec6796b4e10a1b2a598e765fcce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:27.131354 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:27.131688 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:27.131967 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:27.142077 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:27.143532 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106fccb-c79b-41f6-80a7-cc2b127bf41d service glance] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:27.153647 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 735/1475] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:29.529058 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a508-7576-4f08-aea1-057dfd4df76c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:29.529659 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a508-7576-4f08-aea1-057dfd4df76c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:29.529659 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7a508-7576-4f08-aea1-057dfd4df76c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:29.531006 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 741/1476] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:29.536512 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e2a5-080b-489d-abfd-e8c8c33c7cf2 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:29.537034 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e2a5-080b-489d-abfd-e8c8c33c7cf2 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:29.537148 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3e2a5-080b-489d-abfd-e8c8c33c7cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:29.555030 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-4ed3e2a5-080b-489d-abfd-e8c8c33c7cf2 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:29.556235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-4ed3e2a5-080b-489d-abfd-e8c8c33c7cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:29.564215 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 736/1477] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:29.588598 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7a508-7576-4f08-aea1-057dfd4df76c None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:29.601689 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7a508-7576-4f08-aea1-057dfd4df76c None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:29.606199 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:29.606856 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:29.607142 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:29.607419 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] PATH_INFO: `/v3/projects` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:29.608150 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:29.609956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING py.warnings [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:46:29.609956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: warnings.warn(msg) May 25 13:46:29.609956 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: May 25 13:46:29.611109 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7382979-4e01-4680-a7a6-0f459c90966f admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:29.631095 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 742/1478] 10.222.0.21 () {56 vars in 1108 bytes} [Wed May 25 13:46:29 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:46:31.929777 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ff79-cb98-4d8e-a3f9-ed0ff0b0a2a4 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:31.930523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ff79-cb98-4d8e-a3f9-ed0ff0b0a2a4 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:31.930523 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91ff79-cb98-4d8e-a3f9-ed0ff0b0a2a4 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:31.931879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 737/1479] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:31.937808 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2147ce-9d3a-4d38-8d5b-22e468b4b513 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:31.938355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2147ce-9d3a-4d38-8d5b-22e468b4b513 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:31.938355 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2147ce-9d3a-4d38-8d5b-22e468b4b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:31.958552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0e2147ce-9d3a-4d38-8d5b-22e468b4b513 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:31.959795 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0e2147ce-9d3a-4d38-8d5b-22e468b4b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:31.967849 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 743/1480] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:32.012539 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ff79-cb98-4d8e-a3f9-ed0ff0b0a2a4 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:32.028633 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91ff79-cb98-4d8e-a3f9-ed0ff0b0a2a4 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:32.034029 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:32.034674 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:32.034959 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:32.035235 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:32.044458 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:32.046120 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2d88e-c9af-4649-905b-ebc2bf2d141a service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:32.055744 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 738/1481] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:34.490944 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0005-fff0-4939-8940-261c14abdb9c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:34.491494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0005-fff0-4939-8940-261c14abdb9c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:34.491608 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f0005-fff0-4939-8940-261c14abdb9c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:34.492966 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 744/1482] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:34.498413 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c317f1-280e-4856-a2f8-096a5e890600 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:34.498881 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c317f1-280e-4856-a2f8-096a5e890600 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:34.498979 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c317f1-280e-4856-a2f8-096a5e890600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:34.518410 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-c9c317f1-280e-4856-a2f8-096a5e890600 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:34.519621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-c9c317f1-280e-4856-a2f8-096a5e890600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:34.528051 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 739/1483] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:34.570060 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f0005-fff0-4939-8940-261c14abdb9c None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:34.581087 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f0005-fff0-4939-8940-261c14abdb9c None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:34.584758 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:34.585418 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:34.585705 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:34.585983 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:34.601200 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:34.602666 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604719f6-a829-448b-9d62-31976f275480 service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:34.612646 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 745/1484] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:36.971446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa39c2-beb0-46ea-9eb5-6a0f8613869d None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:36.972065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa39c2-beb0-46ea-9eb5-6a0f8613869d None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:36.972065 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa39c2-beb0-46ea-9eb5-6a0f8613869d None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:36.974311 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 740/1485] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:36.980285 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a4eb0-92b3-4910-8f64-e53ad929f50f None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:36.980888 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a4eb0-92b3-4910-8f64-e53ad929f50f None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:36.980888 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a4eb0-92b3-4910-8f64-e53ad929f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:36.999982 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-0b5a4eb0-92b3-4910-8f64-e53ad929f50f None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:37.001239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-0b5a4eb0-92b3-4910-8f64-e53ad929f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:37.009440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 746/1486] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:37.048283 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa39c2-beb0-46ea-9eb5-6a0f8613869d None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:37.060171 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa39c2-beb0-46ea-9eb5-6a0f8613869d None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:37.064009 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:37.064707 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:37.065001 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:37.065279 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:37.076374 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:37.077924 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21827ea9-5f0e-47be-880f-ad7a232f1796 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:37.089452 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 741/1487] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:46:39.574667 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7332b4-6d08-4114-aab7-367328e95b4d None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:39.575259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7332b4-6d08-4114-aab7-367328e95b4d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:39.575259 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7332b4-6d08-4114-aab7-367328e95b4d None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:39.576688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 747/1488] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:39.582986 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b737b8-61d8-4a00-9403-8ce710b6538a None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:39.583511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b737b8-61d8-4a00-9403-8ce710b6538a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:39.583629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b737b8-61d8-4a00-9403-8ce710b6538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:39.602665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-72b737b8-61d8-4a00-9403-8ce710b6538a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:39.603878 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-72b737b8-61d8-4a00-9403-8ce710b6538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:39.611995 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 742/1489] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:39.654904 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7332b4-6d08-4114-aab7-367328e95b4d None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:39.669167 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7332b4-6d08-4114-aab7-367328e95b4d None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:39.672855 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:39.673510 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:39.673792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:39.674065 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:39.684480 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:39.685990 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc38f1-5d82-4aa9-b7f7-c96bbbbca9bc service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:39.697312 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 748/1490] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:42.223955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85570-1dd6-432d-81a0-e53dbf2cace7 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:42.224494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85570-1dd6-432d-81a0-e53dbf2cace7 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:42.224494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85570-1dd6-432d-81a0-e53dbf2cace7 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:42.225817 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 743/1491] 10.222.0.21 () {54 vars in 926 bytes} [Wed May 25 13:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:46:42.231227 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c721b0-2b5b-4717-a0b5-d0fb28d667cc None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:42.231692 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c721b0-2b5b-4717-a0b5-d0fb28d667cc None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:42.231806 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c721b0-2b5b-4717-a0b5-d0fb28d667cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:42.249976 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-44c721b0-2b5b-4717-a0b5-d0fb28d667cc None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:46:42.251194 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-44c721b0-2b5b-4717-a0b5-d0fb28d667cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:46:42.259908 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 749/1492] 10.222.0.21 () {58 vars in 1023 bytes} [Wed May 25 13:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:46:42.300047 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f85570-1dd6-432d-81a0-e53dbf2cace7 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:46:42.311201 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f85570-1dd6-432d-81a0-e53dbf2cace7 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:46:42.314767 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:46:42.315996 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:46:42.316301 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:46:42.316611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:46:42.326124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:46:42.327590 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487322-1951-4214-a551-fb69a86e8449 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:46:42.338551 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 744/1493] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:46:52.244804 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=33 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:46:52.333879 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=33 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:26.068660 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.069789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.069987 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.088657 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:26.089879 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:26.097772 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 750/1494] 10.222.0.21 () {58 vars in 957 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:26.107605 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.108156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.108156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.126429 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:26.127621 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:26.135873 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 745/1495] 10.222.0.21 () {58 vars in 957 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:26.140825 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.152174 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2181619-7b35-4dac-bd5a-e926123730d0 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.156789 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.157445 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.157735 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.158050 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] PATH_INFO: `/v3/domains` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.158788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91825) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 13:48:26.159100 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.160114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING py.warnings [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.160114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: warnings.warn(msg) May 25 13:48:26.160114 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: May 25 13:48:26.161070 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.173963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 751/1496] 10.222.0.21 () {58 vars in 1152 bytes} [Wed May 25 13:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:48:26.182639 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.191936 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751819b8-8e3e-44c2-8918-ad9b4c41d52a None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.197583 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.198242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.198522 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.198798 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] PATH_INFO: `/v3/projects` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.199514 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.201339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING py.warnings [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.201339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: warnings.warn(msg) May 25 13:48:26.201339 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: May 25 13:48:26.202550 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.239854 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 746/1497] 10.222.0.21 () {60 vars in 1152 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:48:26.244761 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.254964 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2398000-edc1-47f9-8df1-cec7584c0514 admin admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.312833 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.313539 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.313827 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.314108 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] PATH_INFO: `/v3/users` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.315165 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.316188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING py.warnings [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.316188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: warnings.warn(msg) May 25 13:48:26.316188 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: May 25 13:48:26.317446 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.340681 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 752/1498] 10.222.0.21 () {60 vars in 1143 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 13:48:26.342537 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.352808 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f210779-7d54-46da-a7ec-f5b4a5003909 admin admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.357365 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.358018 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.358303 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.358580 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] PATH_INFO: `/v3/roles` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.359312 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.360337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING py.warnings [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.360337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: warnings.warn(msg) May 25 13:48:26.360337 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: May 25 13:48:26.361267 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.375214 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 747/1499] 10.222.0.21 () {58 vars in 1121 bytes} [Wed May 25 13:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:48:26.379367 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.390632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72243391-eceb-4dfa-85a1-dd1a0dab2ffc admin admin] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.395319 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.395989 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.396278 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.396581 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] PATH_INFO: `/v3/roles` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.397448 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.398461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING py.warnings [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.398461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: warnings.warn(msg) May 25 13:48:26.398461 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: May 25 13:48:26.399350 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c160-9703-4d56-befc-d79532fedadb admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.412762 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 753/1500] 10.222.0.21 () {58 vars in 1121 bytes} [Wed May 25 13:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 13:48:26.417035 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.427011 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71849488-1028-4c13-8866-b3353f57547f admin admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.433976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.433976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] REQUEST_METHOD: `PUT` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.433976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.433976 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] PATH_INFO: `/v3/projects/532797f07976474c9f594d3acccb4ee8/users/ff941e161f3e451392502a37a792bd6f/roles/ed8e20ce43ae47a49ef4e53d7cf3456d` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.450364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532797f07976474c9f594d3acccb4ee8, user_id=ff941e161f3e451392502a37a792bd6f, role_id=ed8e20ce43ae47a49ef4e53d7cf3456d)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.451418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING py.warnings [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.451418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: warnings.warn(msg) May 25 13:48:26.451418 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: May 25 13:48:26.453426 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1715d91-f129-4660-8630-a6722aa9ac75 admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.468471 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 748/1501] 10.222.0.21 () {60 vars in 1482 bytes} [Wed May 25 13:48:26 2022] PUT /identity/v3/projects/532797f07976474c9f594d3acccb4ee8/users/ff941e161f3e451392502a37a792bd6f/roles/ed8e20ce43ae47a49ef4e53d7cf3456d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:26.480663 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035cb49-d354-4053-9c60-2da7bee6e3ba None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.480960 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035cb49-d354-4053-9c60-2da7bee6e3ba None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.481247 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035cb49-d354-4053-9c60-2da7bee6e3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.509163 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-3035cb49-d354-4053-9c60-2da7bee6e3ba None None] MFA Rules not processed for user `ff941e161f3e451392502a37a792bd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:26.510381 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-3035cb49-d354-4053-9c60-2da7bee6e3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:26.575726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 754/1502] 10.222.0.21 () {58 vars in 957 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:26.589288 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639741-2e09-4ae6-acd8-ac5178608cc6 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.589603 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639741-2e09-4ae6-acd8-ac5178608cc6 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.589884 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639741-2e09-4ae6-acd8-ac5178608cc6 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.591298 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 749/1503] 10.222.0.21 () {54 vars in 963 bytes} [Wed May 25 13:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:48:26.596315 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da164ad-163b-4da9-b20f-f97f7046d292 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.596671 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da164ad-163b-4da9-b20f-f97f7046d292 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.596967 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da164ad-163b-4da9-b20f-f97f7046d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.615759 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-5da164ad-163b-4da9-b20f-f97f7046d292 None None] MFA Rules not processed for user `afc0939b6c0e4e6b81bc8337065cdc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:26.617662 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5da164ad-163b-4da9-b20f-f97f7046d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:26.660811 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:26.669796 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 755/1504] 10.222.0.21 () {58 vars in 1060 bytes} [Wed May 25 13:48:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:48:26.781180 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639741-2e09-4ae6-acd8-ac5178608cc6 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.796466 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639741-2e09-4ae6-acd8-ac5178608cc6 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.805824 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc0939b6c0e4e6b81bc8337065cdc5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.806484 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.806775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.807058 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.816037 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.817526 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:26.826797 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 750/1505] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:48:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:48:26.870140 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06676f24-edda-4ab8-9c1f-c4b8ab020c3c None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.870440 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06676f24-edda-4ab8-9c1f-c4b8ab020c3c None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.870726 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06676f24-edda-4ab8-9c1f-c4b8ab020c3c None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.872007 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 756/1506] 10.222.0.21 () {52 vars in 829 bytes} [Wed May 25 13:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:48:26.875684 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:26.940115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85ad1f-2b6e-46af-bffc-3a753ef38b1c service cinder] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:26.944750 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:26.945627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:26.945914 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:26.946194 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] PATH_INFO: `/v3/users/ff941e161f3e451392502a37a792bd6f` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:26.947409 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff941e161f3e451392502a37a792bd6f)` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:26.948446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: WARNING py.warnings [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:26.948446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: warnings.warn(msg) May 25 13:48:26.948446 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: May 25 13:48:26.949729 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:27.020800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff941e161f3e451392502a37a792bd6f'} {{(pid=91826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.033894 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff941e161f3e451392502a37a792bd6f'} {{(pid=91826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.052895 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff941e161f3e451392502a37a792bd6f'} {{(pid=91826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.053863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff941e161f3e451392502a37a792bd6f'} {{(pid=91826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.054652 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.notifications [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff941e161f3e451392502a37a792bd6f'} {{(pid=91826) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.058689 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:27.071472 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 751/1507] 10.222.0.21 () {58 vars in 1223 bytes} [Wed May 25 13:48:26 2022] DELETE /identity/v3/users/ff941e161f3e451392502a37a792bd6f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:27.076131 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:27.076638 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:27.076754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:27.094574 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:48:27.095832 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:27.163179 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 757/1508] 10.222.0.21 () {58 vars in 957 bytes} [Wed May 25 13:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 13:48:27.176342 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:27.177805 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:27.234840 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be840bc9-fc33-427c-b587-121436de5666 admin admin] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:27.238737 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:27.239393 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:27.239680 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:27.239958 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:27.249742 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:27.251209 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fba08b-4868-4d6c-973a-372dedf129e9 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:27.260921 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 752/1509] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:48:27.611857 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:48:27.613600 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:48:27.625506 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98694-ef54-4688-a9d0-1e0b978a64b9 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:48:27.630494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4118f6e846e744a8a962f12a116b5928', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd26306eca6ff451eb60119f9e4cf5383', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:48:27.631362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:48:27.631644 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:48:27.631919 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] PATH_INFO: `/v3/projects/532797f07976474c9f594d3acccb4ee8` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:48:27.634599 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532797f07976474c9f594d3acccb4ee8)` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:48:27.635773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: WARNING py.warnings [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 13:48:27.635773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: warnings.warn(msg) May 25 13:48:27.635773 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: May 25 13:48:27.637027 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:48:27.701393 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532797f07976474c9f594d3acccb4ee8', 'request_id': 'req-be53a345-ad68-4478-85b2-e39d0335dfa0'} {{(pid=91825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.711209 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532797f07976474c9f594d3acccb4ee8', 'request_id': 'req-be53a345-ad68-4478-85b2-e39d0335dfa0'} {{(pid=91825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.725870 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532797f07976474c9f594d3acccb4ee8', 'request_id': 'req-be53a345-ad68-4478-85b2-e39d0335dfa0'} {{(pid=91825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.737260 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] The token cache is being invalidate because project 532797f07976474c9f594d3acccb4ee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91825) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 13:48:27.737552 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.738494 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.notifications [None req-be53a345-ad68-4478-85b2-e39d0335dfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91825) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 13:48:27.740333 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 758/1510] 10.222.0.21 () {58 vars in 1232 bytes} [Wed May 25 13:48:27 2022] DELETE /identity/v3/projects/532797f07976474c9f594d3acccb4ee8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 13:48:37.256424 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:48:37.733985 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:07.937732 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:07.938372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:07.938372 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:07.940528 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:07.945747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad916e-0267-4ec5-b851-680901e6da46 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:07.946261 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad916e-0267-4ec5-b851-680901e6da46 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:07.946385 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad916e-0267-4ec5-b851-680901e6da46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:07.965201 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-fbad916e-0267-4ec5-b851-680901e6da46 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:07.966400 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-fbad916e-0267-4ec5-b851-680901e6da46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:08.027018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:08.199416 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:08.201158 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:08.280348 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ae16-c0ec-4174-9cf7-00a67343a2f0 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:08.286124 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:08.286781 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:08.287070 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:08.287353 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:08.296253 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:08.297950 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455687dc-207f-4093-ac7c-f45625d18aa5 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:08.308010 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 754/1513] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:08.886186 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc549-a0f2-4e21-abb8-d0780f3009fe None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:08.886417 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc549-a0f2-4e21-abb8-d0780f3009fe None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:08.887286 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebc549-a0f2-4e21-abb8-d0780f3009fe None None] PATH_INFO: `/v3` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:08.888065 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 760/1514] 10.222.0.58 () {54 vars in 925 bytes} [Wed May 25 13:50:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 13:50:08.896004 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d6df-114e-4ae3-941d-74a30cf895fe None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:08.896321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d6df-114e-4ae3-941d-74a30cf895fe None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:08.896629 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823d6df-114e-4ae3-941d-74a30cf895fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:08.916385 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-5823d6df-114e-4ae3-941d-74a30cf895fe None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:08.917632 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-5823d6df-114e-4ae3-941d-74a30cf895fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:08.995792 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 755/1515] 10.222.0.58 () {58 vars in 1015 bytes} [Wed May 25 13:50:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:09.244033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bcd86-2282-45d7-aa52-21a4a572ac47 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:09.244033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bcd86-2282-45d7-aa52-21a4a572ac47 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:09.244033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bcd86-2282-45d7-aa52-21a4a572ac47 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:09.244033 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 761/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:09.249008 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1ebc4-f8eb-4b84-89d2-880b44c90333 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:09.249321 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1ebc4-f8eb-4b84-89d2-880b44c90333 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:09.249717 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1ebc4-f8eb-4b84-89d2-880b44c90333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:09.273985 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-78d1ebc4-f8eb-4b84-89d2-880b44c90333 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:09.275216 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-78d1ebc4-f8eb-4b84-89d2-880b44c90333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:09.284046 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 756/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:09.442322 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bcd86-2282-45d7-aa52-21a4a572ac47 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:09.458542 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bcd86-2282-45d7-aa52-21a4a572ac47 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:09.466891 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:09.467556 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:09.467843 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:09.468123 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:09.478137 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:09.480835 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028c31a-31cc-4495-9547-c003fbc2d0a4 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:09.491702 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 762/1518] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:50:18.070859 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ce9d-5472-4fb0-ac56-f79398ab982f None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:18.071432 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ce9d-5472-4fb0-ac56-f79398ab982f None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:18.071534 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8ce9d-5472-4fb0-ac56-f79398ab982f None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:18.073020 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 757/1519] 10.222.0.21 () {54 vars in 960 bytes} [Wed May 25 13:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:18.078080 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7977-9fae-4bcd-8446-e5c705a70ce7 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:18.078505 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7977-9fae-4bcd-8446-e5c705a70ce7 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:18.078655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d7977-9fae-4bcd-8446-e5c705a70ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:18.097881 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-697d7977-9fae-4bcd-8446-e5c705a70ce7 None None] MFA Rules not processed for user `7280fc8f604e42cdb50973590acaab81`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:18.099107 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-697d7977-9fae-4bcd-8446-e5c705a70ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:18.107622 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 763/1520] 10.222.0.21 () {58 vars in 1057 bytes} [Wed May 25 13:50:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:18.115152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8ce9d-5472-4fb0-ac56-f79398ab982f None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:18.129716 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8ce9d-5472-4fb0-ac56-f79398ab982f None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:18.134866 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7280fc8f604e42cdb50973590acaab81', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:18.135511 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:18.135800 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:18.136081 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:18.145883 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:18.147494 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abee71-725b-45fb-9ef9-82263020cf63 service placement] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:18.158126 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 758/1521] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:20.813049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee1d3-61ae-476f-a109-0f06b0bfb6ee None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:20.813572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee1d3-61ae-476f-a109-0f06b0bfb6ee None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:20.813572 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ee1d3-61ae-476f-a109-0f06b0bfb6ee None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:20.814963 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:20.819791 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7df28-c16c-47dc-9775-223c5138aa2c None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:20.820206 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7df28-c16c-47dc-9775-223c5138aa2c None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:20.820340 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7df28-c16c-47dc-9775-223c5138aa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:20.837830 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-bfd7df28-c16c-47dc-9775-223c5138aa2c None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:20.839013 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-bfd7df28-c16c-47dc-9775-223c5138aa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:20.846726 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:21.005024 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ee1d3-61ae-476f-a109-0f06b0bfb6ee None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:21.020268 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ee1d3-61ae-476f-a109-0f06b0bfb6ee None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:21.025792 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:21.026443 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:21.026730 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:21.027008 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:21.036353 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:21.038049 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221da619-055b-4a8b-ac16-e720c29a3732 service nova] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:21.047862 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 765/1524] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 13:50:21.932346 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f7c3-7c43-4655-bb6f-112d0be4bc47 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:21.932955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f7c3-7c43-4655-bb6f-112d0be4bc47 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:21.932955 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f7c3-7c43-4655-bb6f-112d0be4bc47 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:21.934364 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 13:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:21.939360 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49777b39-f637-4622-a404-685aa3c1498d None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:21.939836 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49777b39-f637-4622-a404-685aa3c1498d None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:21.939940 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49777b39-f637-4622-a404-685aa3c1498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:21.958747 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-49777b39-f637-4622-a404-685aa3c1498d None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:21.959957 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-49777b39-f637-4622-a404-685aa3c1498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:21.967949 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 13:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:22.120482 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0f7c3-7c43-4655-bb6f-112d0be4bc47 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:22.135731 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0f7c3-7c43-4655-bb6f-112d0be4bc47 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:22.141504 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a21198d279bd499fbd1e2f42798f3918', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:22.142156 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:22.142438 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:22.142715 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:22.152584 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:22.154192 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a03e8b-efbc-46ce-88db-9008a28d4993 service nova] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:22.164277 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 761/1527] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:26.216788 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:26.217362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:26.217362 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:26.218725 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 13:50:26.224300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c71a-5369-41d3-a4fd-82f352997200 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:26.224694 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c71a-5369-41d3-a4fd-82f352997200 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:26.224784 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3c71a-5369-41d3-a4fd-82f352997200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:26.243404 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-9ce3c71a-5369-41d3-a4fd-82f352997200 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:26.244627 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-9ce3c71a-5369-41d3-a4fd-82f352997200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:26.252421 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:26.289471 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:26.291191 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:26.350909 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad91a4-6148-4f34-9c22-c6c89f3be0b2 None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:26.354474 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:26.355125 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:26.355416 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:26.355697 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:26.364799 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:26.366245 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abeeb9-7445-4294-bcfa-5f27d2e6090e service neutron] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:26.375595 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 768/1530] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:28.583771 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b99a6-c53a-47f7-9122-4bf45c9e469e None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:28.584300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b99a6-c53a-47f7-9122-4bf45c9e469e None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:28.584300 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b99a6-c53a-47f7-9122-4bf45c9e469e None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:28.585665 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:28.591320 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0008f0-4e8d-441f-b4f1-bc7a4e04ff27 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:28.591754 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0008f0-4e8d-441f-b4f1-bc7a4e04ff27 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:28.592018 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0008f0-4e8d-441f-b4f1-bc7a4e04ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:28.736267 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-5a0008f0-4e8d-441f-b4f1-bc7a4e04ff27 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:28.737521 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-5a0008f0-4e8d-441f-b4f1-bc7a4e04ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:28.745640 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:28.781533 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b99a6-c53a-47f7-9122-4bf45c9e469e None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:28.793189 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b99a6-c53a-47f7-9122-4bf45c9e469e None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:28.796838 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d754f1173e446ceb607dfaeb2231c28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:28.797483 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:28.797775 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:28.798115 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:28.808359 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:28.809863 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c292fc-4984-4980-99bf-48c939ed4ae4 service neutron] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:28.821387 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 764/1533] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:32.699477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.700043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.700043 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] PATH_INFO: `/` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.701443 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:32.706801 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ec31-f5e4-4f3c-a266-6d54f3e10685 None None] REQUEST_METHOD: `POST` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:32.707242 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ec31-f5e4-4f3c-a266-6d54f3e10685 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:32.707455 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2ec31-f5e4-4f3c-a266-6d54f3e10685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:32.725938 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.auth.core [None req-abc2ec31-f5e4-4f3c-a266-6d54f3e10685 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91826) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:32.727135 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-abc2ec31-f5e4-4f3c-a266-6d54f3e10685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:32.734931 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:33.010632 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] Authenticating user token {{(pid=91825) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:33.012330 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:33.071655 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ea087-77bf-4251-8d3b-4b52e2a55ada None None] Validating token access rules against request {{(pid=91825) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:33.075239 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc0939b6c0e4e6b81bc8337065cdc5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91825) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:33.075890 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] REQUEST_METHOD: `GET` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:33.076197 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:33.076477 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:33.086005 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:33.087438 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b14663-22a6-4116-b18d-87e0af30d110 service cinder] RBAC: Authorization granted {{(pid=91825) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:33.092339 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=100 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:33.097693 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 771/1536] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:35.354187 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:35.354790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:35.354790 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] PATH_INFO: `/` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:35.356123 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 13:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 13:50:35.361728 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8047c-d4cc-448d-b3d9-54cb9c5b4c78 None None] REQUEST_METHOD: `POST` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:35.362238 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8047c-d4cc-448d-b3d9-54cb9c5b4c78 None None] SCRIPT_NAME: `/identity` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:35.362348 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8047c-d4cc-448d-b3d9-54cb9c5b4c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91825) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:35.381313 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.auth.core [None req-88f8047c-d4cc-448d-b3d9-54cb9c5b4c78 None None] MFA Rules not processed for user `4118f6e846e744a8a962f12a116b5928`. Rule list: `[]` (Enabled: `True`). {{(pid=91825) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 13:50:35.382531 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG keystone.common.fernet_utils [None req-88f8047c-d4cc-448d-b3d9-54cb9c5b4c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91825) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:35.390643 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: [pid: 91825|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 13:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 13:50:35.671664 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] Authenticating user token {{(pid=91826) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 13:50:35.673375 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.fernet_utils [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91826) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 13:50:35.686649 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9084932-e09e-4605-9ce4-85b26413be16 None None] Validating token access rules against request {{(pid=91826) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 13:50:35.690227 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc0939b6c0e4e6b81bc8337065cdc5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9f61eee6bc284eb2b31fa2d98e2ff62e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91826) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 13:50:35.690870 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] REQUEST_METHOD: `GET` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 13:50:35.691152 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] SCRIPT_NAME: `/identity` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 13:50:35.691430 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91826) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 13:50:35.701162 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 13:50:35.702611 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98402d71-8bf0-4c33-af84-2842cbabce15 service cinder] RBAC: Authorization granted {{(pid=91826) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 13:50:35.713127 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: [pid: 91826|app: 0|req: 767/1539] 10.222.0.21 () {60 vars in 1365 bytes} [Wed May 25 13:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 13:50:45.375688 dv-cinder-803823-8-iscsi devstack@keystone.service[91825]: DEBUG dbcounter [-] [91825] Writing DB stats keystone:SELECT=3 {{(pid=91825) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 13:50:45.707782 dv-cinder-803823-8-iscsi devstack@keystone.service[91826]: DEBUG dbcounter [-] [91826] Writing DB stats keystone:SELECT=99 {{(pid=91826) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}